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| summary results at a glance I 



We self-initiated this audit after a series of 
critical newspaper articles reported on 
concerns raised by the City of West Palm 
Beach (WPB) regarding the cost and 
performance of the Harris Corporation 
(formerly M/A-COM) OpenSky digital 
public safety radio system. Acquisition 
and implementation of the OpenSky 
system has been managed by the 
Municipal Public Safety Communications 
Consortium (MPSCC) established in July 
of 1999 and incorporated as a non-profit 
government organization in April of 2000. 
The goal of the MPSCC was to plan, 
design and implement a county wide 
interoperable municipal public safety 
radio system. 

Better Planning and Project 
Management of Open Sky Could Have 
Improved System Implementation 

While a great deal has been 
accomplished, and many obstacles 
overcome, OpenSky has taken almost a 
decade to become operational since the 
MPSCC was first formed. During that 
time MPSCC membership significantly 
declined from 33 municipalities to the 
current membership of six municipalities 
plus the School District Police. As a 
result, the original goal of having a county 
wide interoperable public safety radio 
system for the majority of municipalities 
has not been fully realized. 

As we reviewed the history of this project 
we identified the following weaknesses in 



the planning and management of 
OpenSky that prevented a timelier and 
more effective implementation: 

Planning 

• Funding for the project was not 
adequately established before moving 
forward. This resulted in a significant 
gap of over five years between 
contract award (2001) and contract 
execution (2006). The lack of 
progress during that period resulted in 
the majority of member municipalities 
withdrawing from the MPSCC. This 
also resulted in over $258,000 in dues 
paid to the MPSCC by municipalities 
that eventually withdrew and received 
no benefit from those payments. 

• System requirements and 
specifications originally established in 
the Request for Proposal (RFP) 
issued in December 2000 were not 
adequately re-evaluated when the 
contract was signed in June 2006. 
One key specification, OpenSky's 
ability to penetrate inside buildings 
(the 12dB 1 attenuation factor), 
remained unchanged. The inability of 
OpenSky to penetrate structures 
exceeding 12dB is a weakness that 
can impact officer and public safety in 



dB (decibels) is a unit used to measure radio signal 
strength. Buildings generally reduce signal strength. 
This is referred to as the attenuation factor. 
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all municipalities on the OpenSky 
system. 

Management 

Implementation of OpenSky has lacked a 
strong project management structure. 
Project manager responsibilities have 
been too fragmented and there was little 
documentation showing that project 
schedules and milestones were 
monitored and adjusted as delays 
occurred. Lack of strong project 
management oversight has led to a 
number of implementation problems 
including: 

• The original target date of March 21 , 
2007 for completion and acceptance 
of Phasel was significantly exceeded. 
The first of the three Phasel 
municipalities, Palm Beach Gardens 
(PBG), did not become operational 
until September 2009 and the second, 
Palm Beach (PB) not until April 2010. 
The third, WPB is still not operational. 
System acceptance occurred on July 
20, 2010 over 40 months after the 
original target date. 

• Acceptance testing processes were 
not always strictly adhered to. Also, 
the final acceptance test required by 
the contract was not completed. 

• System interoperability 2 , a key 
requirement included in the contract, 
was not formally tested and results 
documented prior to system 
acceptance. After we brought this to 
their attention, the MPSCC had their 
consultant conduct a limited test of 
interoperability with the County system 
on August 14, 2012. We observed 



2 The ability of police officers on OpenSky to talk 
directly with officers on the County and other municipal 
public safety radio systems. 



that test and voice communication 
worked. 

• The final acceptance test performed in 
WPB in August 2009, was not well 
planned and managed and no formal 
assessment was conducted to identify 
the exact causes for and resolve the 
problems that resulted in that test 
being terminated. Identifying and 
addressing the specific causes could 
have resulted in a determination of 
whether OpenSky will work in WPB. 

Contracting Practices Used in 
OpenSky Did Not Adequately Protect 
Expenditure of Public Funds 

During our review we identified a number 
of issues with the contracting practices 
followed by the MPSCC including 
concerns with the terms and conditions of 
the M/A-COM (Harris) contract for 
OpenSky. 

The total cost for the OpenSky contract 
was $4,566,000 to cover Phase 1 of the 
system (deployment of OpenSky in PBG, 
PB and WPB). Based on the terms, 
conditions and payment schedule 
established in the contract, MPSCC paid 
$4,002,824 (88%) of the total contract 
price before OpenSky went through full 
acceptance testing. In addition, a clause 
in the contract allowed the vendor to 
deem OpenSky accepted without all 
acceptance tests being successfully 
completed and without formal acceptance 
by the MPSCC. 

In contrast, we found that in a similar 
contract between New York State and 
M/A-COM for a statewide OpenSky 
system, the State was not obligated to 
pay M/A-COM until Phase 1 was 
completed, tested and accepted. 
Considering that this was relatively new 
and unproven public safety radio 
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technology, stronger contract provisions 
should have been in place to better 
protect the MPSCC and its members. 

As previously mentioned, there was a five 
year gap between the award of a contract 
to M/A-COM and the actual execution of a 
contract between MPSCC and M/A-COM. 
Considering the length of time, we 
question whether a new RFP or other 
solicitation should have been issued in 
lieu of signing a contract with M/A-COM 
based on an RFP developed five years 
earlier. 

Finally, we also identified three other 
contract issues: 

• A consulting contract was issued by 
Palm Beach County in the initial 
stages of the MPSCC public safety 
radio project, in which three payments 
totaling $89,353 were made for 
deliverables that could not be 
performed. The three deliverables 
were for installation, testing and 
acceptance; however the payments 
were made four years before a 
contract for a system was signed. 

• A consulting contract was issued by 
the MPSCC with no documented 
deliverables to support the payment of 
$164,510. 

• A purchase order was issued by the 
MPSCC for $85,764 in additional radio 
equipment that should have been 
purchased by a contract "change 
order." 

OpenSky is Performing to Contract 
Specifications in Five Municipalities 

The MPSCC has deployed the OpenSky 
public safety radio system in the 
municipalities of PBG, PB, Atlantis, 
Jupiter and Juno Beach. The system has 



been operational in PBG for over three 
years, in PB for over two and half years 
and in Atlantis for almost a year with no 
major system problems or outages 
reported. Acceptance test results show 
that OpenSky is performing to contract 
specifications. 

In August, 2012, we observed acceptance 
testing prior to deployment of OpenSky in 
Jupiter and Juno Beach and found that 
overall the system performed well. Also, 
interoperability has been achieved with 
the County and other municipalities. In 
addition, the Palm Beach County School 
District Police are currently implementing 
and testing OpenSky under a separate 
contract with Harris. The MPSCC has 
worked effectively with Harris to 
overcome early problems and get 
OpenSky operational in the five 
municipalities. 

Our Survey of Officers Using OpenSky 
Had Mixed Results 

Although OpenSky is operational in five 
municipalities, there appears to be 
performance issues that the MPSCC will 
need to address as it continues to 
manage and maintain OpenSky. In a 
survey we conducted of police officers in 
PBG, PB and Atlantis, over 69% indicated 
that they experienced radio problems that 
could impact officer and public safety. 
Most common among problems cited 
were dead spots, poor audio quality and 
dropped calls. 

These results differ significantly from what 
we observed in the Jupiter/Juno Beach 
test and could indicate the system is 
performing better in some areas than 
others. The MPSCC must address this, 
especially since the types of problems 
reported in our survey could impact officer 
and public safety. 
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Questions Remain Unresolved on 
OpenSky Operability in WPB 

To date, WPB has invested over $5.1 
million in OpenSky. However, the system 
has not been successfully deployed in 
WPB. Plans for deployment were halted 
after significant problems were 
encountered during a final acceptance 
test in August 2009. 

Several enhancements have been made 
to the OpenSky system since that failed 
test in WPB that have improved system 
performance. In addition, a later test 
performed in WPB by an independent 
firm, RCC Consultants, Inc (RCC) in 
February of 201 1 concluded that 
OpenSky performed well except for 
concerns involving indoor coverage in 
certain buildings that exceeded the 12dB 
contract specification. 

However, for OpenSky to meet the needs 
of WPB, and mitigate officer and public 
safety risks, the system needs to perform 
better than current contract specifications. 
This may require additional transmission 
sites at an estimated cost of $440,000. 

WPB has a number of options available to 
them. Based just on the next five years' 
net costs, WPB's least cost option would 
be to move to the County's current 
Motorola analog system. However, as we 
discuss more fully in the details of our 
report, other factors, including future 
costs to migrate to digital technology 
need to be considered in making a 
decision among the various options. 

We were also informed that the MPSCC, 
Harris and WPB are currently discussing 
the option of providing WPB with the 
Harris P-25 IP digital radio system 
integrated with OpenSky. We have not 
seen the cost estimate on this proposal. 



With regard to OpenSky as an option, 
since WPB has never passed a full 
acceptance test, additional testing would 
be needed to determine how well the 
current OpenSky configuration performs 
throughout WPB. 

The MPSCC has Considered Both 
Maintenance and Future Plans for 
OpenSky 

The contract with M/A-COM (Harris) 
OpenSky includes language that requires 
Harris to provide, "full availability of all 
parts, components or comparable parts 
and service for a period of seven (7) 
years on all Seller manufactured 
infrastructure equipment and for five (5) 
years on all subscriber equipment from 
the last date of manufacture." Harris 
Corporation has announced OpenSky2 
as the next generation of the OpenSky 
product line which uses the same radios 
and backbone equipment as OpenSky. 

The MPSCC contracts with the local 
Citation Communications office for 24/7 
OpenSky support. The MPSCC has a 
reserve bank account with a balance of 
$448,821 as of September 30, 2011. 
However, this balance is not based on 
any planning or analysis of the future 
replacement costs for OpenSky and the 
MPSCC will need to begin setting aside 
funds now for eventual system 
replacement. 

Stronger Accounting and 

Administrative Procedures and 
Controls Are Needed 

As part of our review, we identified 
funding of $7.1 million that has been 
received by the MPSCC since its 
inception. We tested expenditures totaling 
$5.1 million to verify funds were properly 
spent and accounted for. Our sample did 
not identify any improper expenditures. 
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In performing the audit work, we noted 
that certain MPSCC's accounting and 
administrative procedures and processes 
were not well developed. MPSCC lacks: 

• Financial statements in conformity 
with governmental accounting 
standards. 

• A formal budget policy and defined 
accounting roles and responsibilities. 

• Inventory tracking controls over capital 
equipment. 

• A travel and expense reimbursement 
policy. 

• A document retention policy. 

• Segregation of duties for payment 
transactions. 

In addition, MPSCC has never had an 
annual audit of their financial statements, 
although it is required by the inter-local 
agreement with their members. 

We also noted, in our sample testing of 
expenditures, five payments totaling 
$32,656 lacked a required second 
authorizing signature, and eleven 
expenditures totaling $55,647 lacked 
evidence of required MPSCC Board 
approval. 

These conditions are due in part 
because, since its inception, MPSCC has 
operated without any full time salaried 
employees, and finance activities have 
been conducted by staff from municipal 
member police departments as collateral 
duties. We do not believe this 
arrangement is sustainable long term. 



More Coordination and Cooperation 
between the MPSCC, the County and 
Municipalities Can Further the Goal of 
County Wide Interoperable Public 
Safety Radio Communications 
During the mid nineties and into the early 
2000's several committees, 

subcommittees and working groups were 
formed to address the radio 
communication needs of the County and 
the municipalities, such as the 
Countywide Public Safety 

Communications Committee (CPSCC) 
and the Communications System and 
Operations Policy Advisory Committee 
(CSOPAC). They have all since been 
disbanded. Meanwhile, the MPSCC has 
implemented OpenSky which, while 
serving far fewer municipalities than 
originally planned, still represents a 
significant investment of both time and 
money in establishing an interoperable 
municipal public safety radio 
communication system. 

The County meanwhile, recently initiated 
a renewal replacement sole source 
procurement to replace end of life 
backbone equipment for their current 
Motorola SmartZone 3.0 analog public 
safety radio system. This purchase will 
enable the County to continue to operate 
their current Motorola analog system for 
the immediate future while also paving 
the way for an eventual conversion to a 
digital public safety radio system. 

Other municipalities, such as Delray 
Beach, Boynton Beach and Boca Raton 
are also facing decisions on the 
replacement of their aging analog public 
safety radio systems. 

As technology advances and all parties 
move forward with plans for enhancing or 
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replacing current radio systems, it would 
benefit the County and its citizens to re- 
establish a public safety radio 
communication committee, with 
representation from the County, 
municipalities and the public safety 
community. 

Such a committee could help ensure 
more cohesive planning and coordination 
that achieves appropriate levels of 
interoperability, while providing the 
various public safety entities flexibility in 
choosing among systems and 
technologies available now and in the 
future. 

Our report contains 18 findings and 26 
recommendations to address those 
findings. 

We received responses from the City of 
West Palm Beach, Palm Beach County, 
and the MPSCC. All three entities 
generally agreed with those 
recommendations for which they were 
responsible for taking action. We 
summarized their responses in the body 
of the report and have included each 
response in its entirety as a separate 
attachment. 
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BACKGROUND 



Planning for a countywide public safety radio system can be traced back to 1990 with 
the issuance of the Kalmanoff Report, a comprehensive study of the County's criminal 
justice system and services related to crime. That report recommended a consolidation 
of police communication services among County and municipal law enforcement 
agencies. Early efforts focused on developing a countywide system that would 
encompass both County and municipal public safety agencies. Ultimately, estimates for 
such a countywide system were deemed too costly. Eventually the County moved 
forward to procure an 800 MHZ Motorola analog radio system for the use of County 
entities. 

The municipalities meanwhile, through the Countywide Public Safety Communications 
Committee (CPSCC), a committee of the Criminal Justice Commission formed in 
August 1994, started planning for a municipal solution. Their efforts led to the creation 
on April 1, 2000 of the not for profit Municipal Public Safety Communications 
Consortium (MPSCC). MPSCC's mission was to provide management, administration 
and technical services necessary to establish a countywide interoperable public safety 
radio system for use by County municipalities. Initial membership for the MPSCC 
included 33 municipalities and the Palm Beach County School District Police. 

The MPSCC utilized a contract issued by the County with L. Robert Kimball & 
Associates (Kimball) to develop a Request for Proposal (RFP) for the procurement of a 
public safety radio system. The RFP titled "Provision of Municipal Voice and Data 
Interoperable Radio Network" was issued on December 7, 2000. On March 23, 2001 a 
letter of award for contract negotiations was issued to TYCO M/A-COM. After a five 
year delay, due to lack of funding, a contract was signed between the MPSCC and 
TYCO M/A-COM on June 22, 2006 for an 800 MHz OpenSky Voice and Data Digital 
Radio Communication System. The contract price was $4,556,000 which covered 
equipment and services for build out of Phase 1 to include PBG, PB, and WPB. Harris 
Corporation (Harris) acquired the contract through acquisition of M/A-COM on May 29, 
2009. 
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OBJECTIVES, SCOPE AND METHODOLOGY 



The scope of our audit of the OpenSky public safety radio system acquisition and 
implementation encompasses both performance and financial objectives. 

The primary audit objectives addressed the following: 

1. Was the Public Safety Radio System procurement and implementation 
adequately planned and managed? 

2. Does OpenSky meet the operability needs of the MPSCC member 
municipalities? 

3. What are the future plans for maintenance and support? 

4. Have all Federal, County and Municipal funds been adequately controlled, 
accounted for and properly expended? 

Our methodology included but was not limited to: 

• An historical review of the Countywide public safety radio project; 

• A review of the contracts associated with the public safety radio project; 

• A review of the implementation planning and management of OpenSky; 

• A review of the OpenSky testing by MA/COM (Harris), the MPSCC and others; 

• A judgmental sample selection based on the financial materiality and testing of 
the MPSCC financials including policy, budget, and expenditures; 

• A survey of the current OpenSky radio system users; and 

• Interviews with MPSCC members, OpenSky technical support staff, and 
professionals/administrators associated with public safety radio systems. 

This audit was conducted in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence obtained 
provides a reasonable basis for our findings and conclusions based on our audit 
objectives. 
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FINDINGS 
AND 

RECOMMENDATIONS 
PLANNING AND MANAGEMENT OF OPEN SKY 



While the MPSCC has gotten OpenSky operational in five municipalities, it has taken 
over a decade to get to this result. In evaluating the history of this project, we identified 
a number of weaknesses in the planning and management of OpenSky that have 
contributed to delays and other implementation problems that the MPSCC has 
experienced. Some of these represent lessons learned that need to be avoided in 
future projects of this nature. 

Finding (1): FUNDING ISSUES WERE NOT RESOLVED BEFORE INITIATING A 
CONTRACT SOLICITATION 



The MPSCC did not have an approved source of funding available when they initiated 
the procurement of a municipal public safety radio system back in 2000. On December 
7, 2000, the MPSCC issued an RFP entitled ""Provision of a Municipal Voice and Data 
Interoperable Radio Network". On February 16, 2001 , two proposals were submitted in 
response to the RFP, one by M/A-COM and the other by Motorola. On March 23, 2001 , 
the MPSCC sent an award letter to M/A-COM advising them of the intent to move 
forward with contract negotiations to purchase the M/A-COM OpenSky radio system. 
The cost of OpenSky in the M/A-COM proposal was between $13.6 million and $17 
million based on the MPSCC membership, which at that time, consisted of 33 
municipalities and the School District Police. 

Documentation we obtained showed that the CJC and subsequently the MPSCC 
planned to fund the project with a surcharge on traffic tickets commonly referred to as 
$12.50 money. 3 The Board of County Commissioners passed resolutions in 1999 and 
again in 2002 to allow for allocation of a portion of the County's $12.50 money to 
municipalities to support their participation in the County's intergovernmental radio 
communication program (ICP). Funds would be allocated to each municipality based on 
that municipality's actual $12.50 collections. The projected $12.50 fund balance for the 
33 municipalities participating in the MPSCC OpenSky system as of October 31, 2001 
was $1.74 million. The MPSCC's funding plan was to use this amount as a down 
payment for the contract. The balance would be funded over ten years using the 
estimated $1.1 million annual $12.50 funds that would accumulate in each MPSCC 
member's municipal accounts. 

However, from the beginning, County administration maintained that the OpenSky 
system does not meet the eligibility requirements for $12.50 money. The MPSCC 
disputes this; however County administration has consistently maintained that 
expenditures for OpenSky are not eligible for $12.50 funds. Without access to $12.50 



3 Per Florida Statute 318.21 (9), $12.50 from each moving traffic violation must be used by the county to fund that 
county's participation in an intergovernmental radio communication program approved by the Department of 
Management Services" 
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money, the MPSCC had no other specific source of funding. As a result, the MPSCC 
could not proceed with negotiating and signing a contract with M/A-COM for the 
OpenSky system until 2006; five years after they issued the award letter. 

During this five year period, as the MPSCC attempted to secure funding, 25 of the 
original 33 member municipalities, as well as the School District Police withdrew their 
membership. This significant drop in membership reduced the size and cost of the 
OpenSky system sufficiently to allow the MPSCC to proceed in 2006 with a contract for 
a Phase 1 build out that would be funded by the three Phase 1 municipalities, PBG, PB, 
and WPB. 4 

Without a firm commitment for funding the MPSCC should not have proceeded with 
contract solicitation and award. Since the five year delay in getting the project started 
resulted in the significant drop in MPSCC membership, the original goal of establishing 
a countywide municipal public safety radio system was diminished. In addition, it 
resulted in some municipalities expending funds for which they received no benefit. As 
the following schedule shows, there were 1 1 municipalities who paid a total of 
$258,559.50 in dues to the MPSCC from 2002 to 2008. These municipalities 
subsequently withdrew from the MPSCC. In accordance with their Interlocal agreement 
they were not entitled to a refund of the membership dues and as a result, they received 
no benefit. 



Membership payment summary from the municipalities which dropped out the MPSCC on or before FY 2008: 

| FY 2002 FY 2003 FY 2004 FY2005 FY2006 FY2007 FY2008 | 
Deposit membership Dues: 

Town of Lantana 4,000.00 4,000.00 4,000.00 4,950.00 4,950.00 4,950.00 6,985.00 

Village of North Palm Beach 4,000.00 4,000.00 4,950.00 4,950.00 4,950.00 

Boynton Beach 15,000.00 15,000.00 15,000.00 18,562.50 18,562.00 

South Palm Beach 4,000.00 

Palm Springs 4,000.00 

Manalapan 4,000.00 

Tequesta 4,000.00 

Boca Raton 15,000.00 

Lake Clarke Shores 4,000.00 

Greenacres 10,000.00 10,000.00 10,000.00 12,375.00 

Lake Worth 10,000.00 10,000.00 10,000.00 12,375.00 Grand Total: 

Total 78,000.00 43,000.00 39,000.00 34,650.00 28,462.50 28,462.00 6,985.00 258,559.50 

Recommendations: 



1) The MPSCC should ensure that sufficient funding exists before proceeding 
with any procurement and especially before awarding a contract. 



4 After the contract was signed in 2006 additional membership changes occurred resulting in the current MPSCC 
membership of 6 municipalities. Also, the School District Police rejoined the MPSCC in 2010 and is implementing 
OpenSky under a separate contract with Harris Corporation. 
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Management Response: MPSCC 



1) The MPSCC will continue to provide the proper procurement practices by 
requiring funding to be established prior to purchase or contract for purchase. 

Finding (2): SYSTEM REQUIREMENTS AND TECHNOLOGY CHANGES WERE 
NOT ADEQUATELY RE-EVALUATED FIVE YEARS AFTER THEY WERE FIRST 
ESTABLISHED 



As we discussed above, over five years elapsed between the issuance of the RFP in 
December, 2000 and the signing of a contract with M/A-COM for OpenSky in June, 
2006. Considering that length of time, we believe the MPSCC should have re-evaluated 
system requirements and issued a new RFP to determine if additional vendors, more 
advanced technology, or both were available to meet their members' needs. Also, 
MPSCC membership had substantially changed, from 33 municipalities when the RFP 
was issued, to 8 when the contract was signed. During that five year period there were 
also significant changes occurring to building structures and building codes. 

One key contract specification that was left unchanged from the original proposal was 
the requirement to penetrate buildings of a certain density (attenuation factor of 12dB). 
The RFP issued in December 2000 categorized building structures as Light (8dB 
attenuation factor), Medium (12dB attenuation factor) or Heavy (20dB attenuation 
factor). The RFP specified a general coverage requirement of Medium Buildings 
throughout the county except for two Special Coverage Areas. Amendment #3 of the 
RFP clarified the Special Coverage area in that it recognized that certain "heavy" 
buildings such as hospitals, banks, condominiums, etc. may not have the required 
interior coverage and asked vendors to propose various solutions to this problem. The 
contract that the MPSCC signed with M/A-COM in 2006 did not address Special 
Coverage areas previously identified in the original RFP. 

The Special Coverage areas identified in the original RFP along with changes in 
infrastructure that occurred over the intervening five year period, further suggest that the 
MPSCC should have considered reaching out to vendors through a Request for 
Information (RFI) 5 in order to obtain current information regarding technology advances 
and building/structure density. In lieu of the RFI, the MPSCC should have re-issued the 
RFP. Officials with the MPSCC indicated that they were unaware of any new 
technology by Motorola or other vendors and therefore did not see any reason to issue 
a new RFP in 2006. 

Because the radio system was purchased using a revised negotiated proposal by one 
vendor, it is unknown if an RFP or RFI could have produced a more technologically 
improved radio system and/or a better price for a radio system. In addition, leaving the 
specification for in-building penetration unchanged at 12dB has resulted in insufficient 
coverage in denser buildings. This needs to be resolved as OpenSky is being been 
built out, as we have recommended in other sections of this report. 



5 A request for information (RFI) is a standard business process whose purpose is to collect written information about 
the capabilities of various suppliers. Normally it follows a format that can be used for comparative purposes. 
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Finding (3): IMPLEMENTATION DID NOT INCLUDE PLANS FOR RADIO 
COMMUNICATION FOR NON-LAW ENFORCEMENT OPERATIONS SUCH AS 
PUBLIC WORKS, UTILITIES AND TELEMETRY DATA 



The contract for the MPSCC OpenSky radio system was focused on public safety radio 
communications and did not directly address other radio communications such as non- 
police voice communication and/or analog telemetry data communications. These other 
two-way radio communications needs for existing or continued legacy system 
requirements should have been included in the implementation of the new radio 
communication system. 

With the exception of WPB, MPSCC members' needs for public safety (police) radio 
communications has been addressed by the selected OpenSky radio system. This 
includes the implementation of OpenSky in the PBG, PB, Atlantis, Jupiter, and Juno 
Beach. However, there is a question of how well all the radio communication needs of 
the WPB can be met by OpenSky. 

WPB utilizes its current Motorola SmartNet radio system for public safety (police), public 
utilities (voice and telemetry data), public works, parks and recreation, and volunteer 
groups such as Citizens on Patrol. The focus of implementation of OpenSky in WPB 
has been for their Police Department radio communication needs. Although possible 
solutions to non-police communication needs may have been discussed, no plans to 
solve these issues within the OpenSky configuration have been formalized or agreed 
upon between WPB and the MPSCC. Since implementation of OpenSky was intended 
to replace the existing Motorola SmartNet system, without a cost effective solution for 
their non-police radio communication needs, OpenSky may not adequately meet all the 
radio communication needs of WPB. 

When we discussed this with the MPSCC, they indicated that each municipality was 
responsible for identifying their individual radio subscriber unit needs and purchasing 
those subscriber units outside of the MPSCC contract. We acknowledge that this has 
been the process that has been followed in acquiring subscriber units needed to operate 
on the OpenSky system. However, since the MPSCC was tasked with the mission to 
plan, design and implement a municipal public radio system for all its members, such 
planning should have included addressing all of their radio needs that would be 
impacted by the MPSCC system. This is especially important for WPB since they have 
a significant non-law enforcement public radio system configuration operating on their 
current Motorola SmartNet system that OpenSky was designed to replace. 

Recommendations: 



2) The MPSCC and WPB management should work together to determine whether 
OpenSky can provide the most cost effective solution for WPB's non-public 
safety radio communication needs. 
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Management Response: MPSCC 



2) The MPSCC acknowledges this recommendation, and agrees that West Palm 
Beach's radio system needs, both public safety and nonpublic safety, need to be 
identified and a comprehensive solution provided. 

Finding (4): IMPLEMENTATION OF OPENSKY LACKED STRONG PROJECT 
MANAGEMENT 



"The Project Manager is the person responsible for ensuring that the Project Team 
completes the project. The Project Manager develops the Project Plan with the team 
and manages the team's performance of project tasks. It is also the responsibility of the 
Project Manager to secure acceptance and approval of deliverables from the Project 
Sponsor and Stakeholders. The Project Manager is responsible for communication, 
including status reporting, risk management, escalation of issues that cannot be 
resolved in the team, and, in general, making sure the project is delivered within budget, 
on schedule, and within scope." 6 

OpenSky project management was outsourced to Kimball, the MPSCC's 
implementation contractor. Kimball then assigned responsibility to a third party project 
manager from General Dynamics Corporation. In addition, both M/A-COM (Harris) and 
Citation Communications had project managers assigned to the planning and 
implementation of OpenSky. 



MPSCC OpahSky R«tJlo NGIwbrK 
Program ScritHlLiU (ProlImJniry) 



EttTS 
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The only project 
management 
documentation we 
could find was the 



We found little documentation of project schedule changes, milestones date tracking, 
issue (problem) resolution, task assignments, or regular status reports by the Kimball 
assigned project manager. It appears that without the adequate project management 
support for the 
OpenSky project, 
local technical staff at 
PBG and members of 
the MPSCC took on 
various project 
management duties 
and responsibilities 
although no one was 
specifically 
designated as having 
overall project 
management 
responsibility. 



1 




Figure 1 



http://www2xitxornell.edu/computer/robohelp/cpmm/Project_Roles_and_Responsibilities.htm 
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original contract project schedule (Figure 1) and a plan to implement OpenSky in WPB 
following the RCC test in February of 2011 (Figure 2, Page 31). The original contract 
plan for OpenSky showed implementation of Phase 1 , which included PBG, PB, and 
WPB, by March of 2007. This document was "Exhibit D" in the original contract 
between the MPSCC and M/A-COM and is dated January 30, 2006. Those milestone 
dates were significantly exceeded. Acceptance testing did not begin until July of 2009 
and WPB, a Phase 1 city, is not yet operational. System acceptance occurred on July 
20, 2010, over 40 months after the contract project schedule date. 

In addition to significant delays in meeting the original 2007 milestone dates, there were 
a number of other problems we identified in carrying out and/or documenting various 
steps in the contract Acceptance Test Plan (ATP). A number of these problems 
ultimately contributed to the poor performance of the OpenSky system in the failed test 
conducted in August of 2009 in WPB. Following is a summary of those acceptance test 
issues: 

RF Coverage Issues In WPB Were Identified But Not Fixed Before 
Proceeding With Acceptance Testing. 

As a prerequisite to executing the formal ATP for Phase 1, M/A-COM (Harris) 
first performed an RF Integrity test to determine if there was sufficient radio 
frequency (RF) signal strength for on-street (outdoor) "talk out" and "talk in" as 
well as in-building (indoor) "talk out" and "talk in" in the planned coverage areas 
for Phase 1 , PBG, PB, and WPB. The RF signal coverage maps showed that 
there was sufficient signal strength to proceed with acceptance testing of the 
OpenSky radio system. However, later it was discovered that the westernmost 
part of WPB known as Ibis had not been included in the RF Integrity map 
planned coverage area. In discussions with the MPSCC they indicated that this 
was an oversight. It was not clear how this omission occurred and why it was not 
identified when the RF maps were drawn in February of 2006. Better project 
management oversight might have resulted in recognizing and addressing this 
significant omission in coverage for WPB sooner. 

Despite subsequently identifying this coverage omission, the MPSCC, their 
consultant and the vendor agreed to proceed with the radio system testing, 
including outdoor and indoor RF coverage and voice quality testing (DAQ 7 Test) 
before resolving the issues in Ibis. 

The tests were conducted in all three Phase 1 municipalities, PBG, PB, and 
WPB, in July 2009. The test results showed that the system passed the DAQ 
test outdoor (840 grids tested) with 97.74% coverage and indoor (337 grids 
tested) with 97.03% coverage. However, the indoor test for western WPB, Ibis, 
was considered a "Not Tested by Mutual Agreement" area and not counted in the 
test result calculations. Based on the exclusion of Ibis from the DAQ indoor test, 
the MPSCC and Harris agreed that the indoor DAQ test was a failure for WPB. 
Although the DAQ test did not pass in WPB and the results were not accepted 



7 DAQ is Delivered Audio Quality. 
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and signed off on by the MPSCC and their consultant, the MPSCC and Harris 
agreed to move forward with additional parts of the acceptance testing. 

We do not believe that the MPSCC should have agreed to proceed beyond the 
DAQ test until the coverage issues in WPB were resolved and the system 
passed the DAQ test. As we discuss later in this report, this became a significant 
problem when the final sequence in the acceptance test plan, the Reliability Test, 
was attempted and failed in WPB. 

Acceptance Testing Procedures For the RF Coverage Test May Not Have 
Been Properly Followed. 

One of the key tests in the acceptance testing sequence for any public safety 
system is the Radio System RF Coverage test which includes DAQ testing. This 
test, in addition to measuring the strength of the signal received by the radios 
throughout the predicted coverage area, measures the quality of the voice 
communication both talk out (from dispatch to the radio unit) and talk in (from the 
radio unit to dispatch). A sufficient number of locations (grids) are tested both 
outdoor and indoor to verify coverage throughout the required coverage area. A 
specific criterion is established for scoring the voice quality on a scale of 1 to 5. 
A voice quality score of 3.4 or higher is required to pass for each test point. 

As previously discussed above, the DAQ test for OpenSky was conducted in July 
of 2009 covering all three Phase 1 municipalities, PBG, PB and WPB. The test 
was performed over four days starting on July 9 th . The test results showed that 
for outdoor testing, 840 grids were tested with 821 scored as passing (97.74%). 
Indoor tests showed 327 out of 337 grids passing (97.03%). 

As part of the documentation we reviewed for this test, we obtained a document 
marked "Confidential Kimball Use Only." This document represented notes 
taken by one of the Kimball consultants who participated in the DAQ test. This 
document contained a number of observations that seemed to question the 
procedures followed during testing, as well as the pass/fail scores being given. 
The document also noted numerous radio performance problems. Examples of 
observations made by the Kimball consultant include: 

"I observed that, in most transmissions, [dispatch] only gives a short reply 
and not a few seconds of audio, which is what the test requires" 

"Overall, the grid is being considered as either "pass" or "fail" without 
regard to the scoring levels used in the ATP" 

"In my opinion, the system would not pass as there is a great deal of 
digital echo... audio dropouts, and partially broken transmissions 
throughout the area" 

Overall, I would estimate that more than 50% of the grids have less than 
DAQ 3.0, but some of these grids are outside the jurisdiction" 
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"Also, in-building coverage is questionable.. .Given that this system is used 
for law enforcement, I am not comfortable with the overall coverage and 
quality" 

The Kimball consultant's observations taken as a whole appear to contradict the 
recorded tests results showing a pass rate above 97% for both outdoor and 
indoor DAQ testing. We saw no documentation from either the MPSCC or 
Kimball on how these observations were resolved. We discussed the Kimball 
document with the MPSCC and they indicated that some of the radio problems 
identified resulted from grid testing that was performed beyond the required 
coverage area to establish the outer boundaries of the system. They also stated 
that problems identified during that DAQ test were addressed and resulted in a 
number of significant improvements to the OpenSky system, many provided by 
Harris at no cost to the MPSCC. 

We agree that the MPSCC has been effective in working with Harris to make 
substantial hardware and software upgrades that have improved system 
performance since this 2009 test. However, we also believe that the Kimball 
consultant's observations bring into question whether the recorded DAQ test 
results accurately reflect the system's performance at that time and whether the 
ATP test protocols were followed. Having recorded acceptance test results that 
show the system performing better than it actually is, even if the problems are 
eventually fixed, puts the buyer at risk for accepting a system that could 
subsequently require costly improvements to bring performance up to contract 
requirements. Having a stronger project management structure in place would 
help guard against any deviations from acceptance testing requirements or 
scoring of tests results that may not accurately reflect actual test results. 

The Final Reliability Test Performed in WPB Was Not Well Planned and 
Managed. 

Despite the fact that the July 2009 DAQ test was considered a fail for WPB, due 
primarily to lack of coverage in Ibis, the vendor and the MPSCC agreed that 
testing procedures should continue on to the final step in the acceptance testing 
sequence, the System Reliability test, per the contract's ATP for WPB, PBG and 
PB. A system reliability test was scheduled for WPB for August 31 , 2009. 

The "system reliability test" is described in the ATP as: 

The fourteen (14) day Reliability Test will be conducted once the 
Acceptance Tests listed above (Sections 1-13) have been approved by 
both the Seller and Buyer and is subject only to System Failure scenarios 
as defined in Table 1 below. The loading of the system will be comprised 
of no more than eight (8) users, being a mixture of 4 mobiles and 4 
portables. 
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Failures shall be defined as follows: 

Major System Failure - A Major System Failure shall be defined as 
occurring when the system experiences one of the failures of Seller- 
supplied hardware described in Table 1 which compromises the system's 
ability to operate as a wide area trunked radio system. 

Minor System Failure - A Minor System Failure shall be defined as 
occurring when the system experiences one of the failures of Seller- 
supplied hardware described in Table 1 that is considered minor in nature 
and has no material effect on the overall operation of the system or major 
system components. 

Table 1 - System Failure Definitions 



Failure Description Major Minor 



a) 


Failure of System Control Equipment 


X 




a1) 


Failover from Primary to Standby (SCE) 




X 


a2) 


Failure of Primary & Standby (SCE) 


X 




a3) 


Failure of Network Administration Server 




X 


a4) 


Failure of Regional Network Manager 




X 


b) 


Failure of Site Control Equipment 


X 




c) 


Failure of Two (2) or more Site Base 




X 


d) 


Failure of < One (1 ) Site Base Stations 




X 


e) 


Failure of Cell Site 




X 


f) 


Failure of Two (2) or more consoles 


X 




g) 


Failure of one console 




X 


h) 


Failure of same or like item of backbone 


X 






equipment > Three (3) times during 






Reliability Test period 



We found little documentation related to this test. However, from our interviews 
of individuals involved in various aspects of the test, we concluded this WPB test 
did not follow the test protocol described above. The WPB test used a test 
configuration that included a combination of Motorola and Harris equipment. 
Radio communications were transmitted through the OpenSky Gateway using 
both Harris and Motorola radios communicating to dispatch operators manning 
WPB's Motorola SmartNet dispatch consoles. A total of 40 officers were included 
in the test using some combination of Harris and Motorola portable and mobile 
radios. 

During the test WPB officers were consistently reporting numerous radio 
communication failures and after three days, on September 2, 2009, the WPB 
test was cancelled over concerns for officer and public safety. 

We could not clearly establish who, if anyone had overall responsibility for the 
planning and execution of this test. However, it was not controlled and managed 
by the MPSCC or their consultant Kimball, who was responsible for project 
management for MPSCC. According to one official with the WPB Technical 
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Team, they were not responsible for organizing or conducting the test although 
they did monitor some of the results. Officials within the WPB Police Department 
had various roles related to assembling, prepping and assigning out radios to the 
officers participating in the test but we were not able to establish whether anyone 
from WPB had overall responsibility for planning and managing the test. Again, a 
lack of sound project management by the MPSCC, with one individual having 
overall responsibility, resulted in this test being poorly planned and executed. 

The West Palm Beach Test Results Were Not Adequately Documented and 
Evaluated 

There were many reasons presented by Harris, the MPSCC, and WPB as to why 
the reliability test failed in WPB. These included: 

• How the hardware and software systems were configured, 

• How the users operated in a mix of the analog Motorola equipment and 
digital OpenSky equipment, 

• Questions about whether Harris radios, which had been in storage, were 
properly prepped and batteries charged, and 

• Inherit problems with the OpenSky system. 

However, we were not able to locate any documentation that recorded the results 
of this test even for the abbreviated three day period that it was run. Also, as a 
follow-up to the abrupt termination of the test, there was no "root cause analysis" 
or after action report done by the MPSCC to make an informed, factual 
determination of the underlying cause(s) for why the OpenSky system performed 
so poorly. The system's performance in this test was especially puzzling given 
the results of the RF Coverage/voice quality (DAQ) test that was done less than 
two months earlier that showed over a 97% pass rate for both outdoor and indoor 
coverage. 

The WPB Technical Team produced two reports critical of OpenSky subsequent 
to the August 2009 test, one dated November 08, 2009 and another dated 
September 20, 2010. Those two reports raised serious doubts among WPB 
officials about the viability of OpenSky to operate in WPB and the issue has 
remained unresolved to this date. The MPSCC should have done more to 
identify the causes for the poor performance of OpenSky in the August 2009 test. 
This could have resulted in a more timely resolution on whether OpenSky can 
work in WPB. This was another outcome of not having strong centralized project 
management in place. 

Steps were skipped in the acceptance testing plan 

Reliability Test: Despite the fact that the Reliability Test in WPB failed and was 
terminated after three days, the MPSCC decided to continue on with 
implementation of OpenSky in both PBG and PB. Without a formal reliability test, 
PBG went live with the OpenSky system on September 1 , 2009 and PB went live 
on April 1, 2010. Therefore the OpenSky system never passed a full formal 14 
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day reliability test with documented test results. In reviewing the contract 
acceptance test documentation, we also noted that the fourteen step 
"Acceptance Test Procedures" which include the Reliability test were never 
signed-off by the MPSCC or their consultant/project manager Kimball. 

Interoperability Test: The MPSCC RFP states, "The overall goal of this project is 
to develop the means to bring together all of the public safety agencies operating 
in Palm Beach County under one communications umbrella which will provide 
instant direct voice communications and data sharing." One of the objectives 
developed by the MPSCC was to "Provide countywide interoperable voice and 
data communications with seamless user roaming via one radio network between 
the thirty member municipalities, County, State, Federal and Military agencies." 

The MPSCC contract, dated January 27, 2006, included in Section 9 of the 
Acceptance Test Procedures the testing of "Interoperability". However, when we 
requested documentation to support that interoperability of OpenSky was tested 
the MPSCC was unable to provide it. Subsequently as we were nearing the 
completion of our audit, the MPSCC staff, Citation Communications and Kimball 
performed and documented a test of OpenSky interoperability with the County 
Motorola system, through the OpenSky gateway. We observed this test. The 
test successfully established two way voice communication with a PBSO officer 
on the County Motorola system. 

Recommendations: 



3) WPB should work with the MPSCC to plan and schedule another System 
Reliability test utilizing the standard Harris equipment configuration. 

4) The MPSCC needs to ensure for future acquisitions or any expansion of the 
current OpenSky system, that all acceptance testing required by contract is 
completely and fully documented. 

Management Response: MPSCC 

3) A proposal for a P25 system was received by the City of West Palm Beach. 

4) The MPSCC agrees and will arrange for any future testing to include more 
extensive and comprehensive documentation. 

Management Response: WPB 

3) A proposal for a P25 system was received by the City. 
OIG Comment: 



If the MPSCC and WPB move forward with acquisition of a P25 system, 
recommendation 3 will no longer be applicable. A new acceptance test plan will 
need to be developed, performed and documented as part of the acquisition of a 
P25 system. 
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CONTRACTING PRACTICES 



During our audit we identified a number of issues with contracting practices followed by 
the MPSCC including concerns with the terms and conditions of the M/A-COM (Harris) 
contract for OpenSky. 

Finding (5): THE CONTRACT PAYMENT SCHEDULE COULD HAVE BEEN 
CONSTRUCTED TO PROVIDE MORE PROTECTION FOR THE EXPENDITURE OF 
PUBLIC FUNDS 



The total cost for the OpenSky contract was $4,566,000 to cover Phase 1 of the system 
(deployment of OpenSky in PBG, PB, and WPB). Based on the terms, conditions and 
payment schedule established in the contract, MPSCC paid $4,002,824 (88%) of the 
total contract price to the vendor before OpenSky was subjected to acceptance testing. 

In contrast, under the terms of a contract that was executed between the State of New 
York and M/A-COM for OpenSky, the State was not obligated to pay M/A-COM until the 
Primary Region (Phase One) was completed, tested and accepted. Ultimately, the 
system was unable to pass Phase One acceptance testing and the contract was 
terminated. Considering that this was relatively new and untested technology, at a 
minimum the MPSCC could have established a contract payment schedule that would 
have provided for a more substantial portion of the total contract price to be paid upon 
successful passage of all acceptance testing. 

We acknowledge that as the system was built out and went through acceptance testing, 
the MPSCC was successful in working with Harris to fix problems, upgrade hardware 
and software and tune the system to perform to contract specifications. However, the 
original contract was with M/A-COM and therefore, it is unknown whether the MPSCC 
would have achieved the same level of success working with M/A-COM without the 
leverage of a more substantial payment held pending successful completion of 
acceptance testing. 

Recommendations: 



5) For all future contracts MPSCC should ensure contract terms and conditions 
are sufficient to adequately protect public funds. 

Management Response: MPSCC 



5) In the future, milestones in MPSCC contracts will provide for more extensive 
validation of functionality, both in the development of the milestone and in its 
essential value based on the dollar amount of the milestone payment. 
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Finding (6) ACTIONS TAKEN BY THE MPSCC RESULTED IN ALLOWING THE 
VENDOR TO INVOKE THE BENEFICIAL USE CLAUSE 



On July 20, 2010, Harris Corporation sent a letter to MPSCC requesting final contract 
payment. In the letter, Harris advised the MPSCC that PBG and PB had successfully 
transitioned to the OpenSky system, and therefore since they were operationally using 
OpenSky for "beneficial use," it was considered accepted by the buyer. 

This letter was written based on the Beneficial Use clause of Section 13.2 of the 
contract which states: 

"Notwithstanding the Acceptance Testing of the System set forth in Section 13.1 
above, if Buyer commences use of any portion of the System for its intended 
purpose, other than for the express purpose of training or testing as mutually 
agreed upon by Seller and Buyer in writing, prior to System Acceptance it shall be 
considered as being for the "Beneficial Use" of the Buyer and the applicable portion 
of the System shall be deemed accepted by Buyer. Seller must provide written 
notice to Buyer of any portion of the System as being for "Beneficial Use", 
specifying which portion of the System as being for "Beneficial Use" or said 
"Beneficial Use" will be waived until the date of such written notice to the Buyer 
from the Seller." 

On August 26, 2010, the MPSCC Board met and approved payment of the final invoice 
for the Harris contract. On September 9, 2010, the MPSCC issued check number 141 
in the amount of $553,176 to the Harris Corporation. The amount of the check 
represented payment of $325,000 for System Acceptance in accordance with section 
9.1.5 of the contract and payment of $228,176 for Final Payment in accordance with 
section 9.1 .6 of the contract. 

While this clause is not uncommon in contracts such as this, the MPSCCs decisions to 
deploy OpenSky in PBG and PB before resolving performance issues in WPB did not 
appear to take into account the implications of this contract term. Allowing Harris to 
deem the system accepted and receive final payment while the largest Phase 1 city, 
WPB, was not operational and had not passed the final acceptance test, left the 
MPSCC with little or no leverage to get Harris to address the problems. 

Finding (7): PAYMENTS WERE MADE FOR SERVICES OUTSIDE OF THE SCOPE 
OF A CRIMINAL JUSTICE COMMISSION CONTRACT 



To initiate the project, the Criminal Justice Commission (CJC) entered into a consulting 
contract with L. Robert Kimball and Associates, Inc. (Kimball) on May 2, 2000 for 
$1 85,000. The scope of the contract was for "Support Services for the Municipal Public 
Safety Communications Radio Project" and included the following professional service 
payment milestones as referenced in Exhibit "C" of the contract: 

Milestone 1 : Completion of Phase I 1 0%=$1 8,850 

Milestone 2: RFP Release 20%=$37,700 
Milestone 3: Selection of Vendor 20%=$37,700 
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Milestone 4: Beginning of System Installation 
Milestone 5: Beginning of System Testing 
Milestone 6: System Acceptance by County 



20%=$37,700 
20%=$37,700 
10%=$1 8,500 



The contract stated: 

Article 3 - Payments to the Contractor, Sub Section A states in part "The 
contractor will bill the county at the amounts set forth in Exhibit C for services 
rendered toward the completion of the Scope of Work/Services." Article 3 - 
Payments to the Contractor, Sub Section B states "Invoices received from the 
CONTRACTOR pursuant to this Contract will be reviewed and approved by the 
COUNTY'S representative, indicating that services have been rendered in 
conformity with the Contract and then will be sent to the Finance Department for 
payment." 

Our review of documentation showed that payments totaling $131,950 were made for 
completion of the first four milestones. Subsequently, a final payment was made on 
November 20, 2001 in the amount of $51 ,653 for "final work completion" and the total 
amount paid on this contract was $183,603. As the above schedule shows, milestones 
#4, 5 and 6 were for system installation, system testing and system acceptance. 
However, as previously mentioned, due to delays in obtaining funding, a contract to 
purchase the OpenSky system was not executed until June, 2006. Therefore those 
final three milestone tasks could not be performed. 

Correspondence we obtained dating back to 2001 between the then Director of CJC 
and Kimball indicated that Kimball performed other services such as research, 
attendance at meetings and educational presentations. However, the contract was 
never amended to reflect that significant change in the scope of services or a 
determination made whether those services should have been paid at the same rate as 
the deliverables established in the contract. As a result, Kimball was paid $89,353 in 
2001 for three milestone deliverables that were not performed. 

Recommendations: 



6) When contract deliverables are changed during the execution of a contract, a 
contract amendment or change order must be issued and a determination made 
whether contract costs should be adjusted. 

Management Response: MPSCC 



6) The MPSCC agrees that change order forms need to be executed. Also, 
contract changes will continue to be taken before the Board of Directors for 
discussion and approval. 
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Finding (8): CONTRACT DELIVERABLES WERE NOT ADEQUATELY 
DOCUMENTED 



On May 22, 2006 the MPSCC entered into a contract in the amount of $164,510 with L. 
Robert Kimball and Associates for the provision of telecommunications and related 
technology services. The contract provided thirteen (13) tasks or provisions of 



services. These Included: 


Taskl 


MPSCC Advocate and General Consulting Services 


Task 2 


Review of M/A-COM Contracts & Equipment Lists Comprehensive System 




Design Review 


Task 3 


Project Kick-Off Meeting 


Task 4 


FCC Licensing and frequency Coordination 


Task 5 


Contract Compliance 


Task 6 


System Staging 


Task 7 


interoperability System design and implementation 


Task 8 


Implement the Migration/Cutover Plan 


Task 9 


Final System Acceptance Testing 


Task 1 


System Coverage Acceptance Testing 


Task 1 1 


Training 


Task 12 


System Documentation 


Task 13 


Project Close out 



Payments on the contract were made beginning on October 5, 2006 and the final 
payment was made on August 12, 2010. During the course of this audit, requests were 
made by the OIG to the MPSCC and L. Robert Kimball and Associates regarding the 
deliverables by task as outlined in Exhibit "A" of the contract. Neither organization was 
able to provide documentation to support satisfaction of these deliverables. As a result, 
the OIG was unable to substantiate the delivery of the services as contracted. 

Recommendations: 



7) The MPSCC must ensure that they receive adequate supporting documentation 
of all contract deliverables before contract payments are made. 

Management Response: MPSCC 



7) The MPSCC agrees with this recommendation. 

Finding (9): MPSCC FAILED TO ISSUE A CONTRACT CHANGE ORDER 



The MPSCC and M/A-COM (Harris) entered into a System Purchase Contract in the 
amount of $4,556,000 on June 22, 2006 for the provision of "Multiple Voice and Data 
Interoperable Radio Network". M/A-COM was responsible to design, furnish, deliver, 
and install the hardware and software for the System and provide the documentation, 
deliverables and services in accordance with the terms of the contract. 
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On December 3, 2007 a payment in the amount of $85,764.15 was made for additional 
equipment to relocate one of the two network switching centers for the purpose of 
system redundancy. This payment was made via MPSCC Purchase Order (07-100) 
dated April 1, 2007 and outside the contract terms and conditions of the contract 
between the MPSCC and M/A-COM (Harris) corporation. The contract was not 
amended to provide additional dollars for additional services or an increased scope of 
service. The contract provides in Section 8.1 .1 that: 

Section 8.1 .1 of the contract states: "In the event of any change in the Hardware 
after the approval by Buyer and Seller of the Detailed Equipment List, Seller shall 
prepare and submit a request for Change Order in accordance with the 
procedures set forth in this section requesting an equitable adjustment in the 
price to reflect any added cost and expense of such change." 

Recommendations: 



8) All MPSCC purchases made for additional OpenSky Hardware and/or Software 
should follow the terms of the contract with M/A-COM (Harris) and be purchased 
by a contract Change Order. 

Management Response: MPSCC 



8) The MPSCC agrees with this recommendation. 



THE OPERABILITY OF OPEN SKY 



The MPSCC has overcome a number of challenges in implementing OpenSky. Funding 
uncertainties and a shrinking membership base represent two of those challenges. 
However, perhaps the biggest challenge has been implementation of the digital radio 
technology itself. Most experts agree that digital radio systems are the future for public 
safety radio. They offer many advantages over older systems including more efficient 
use of radio frequencies. In the wake of September 1 1 , 2001 , the Federal government 
in an effort to promote interoperability has encouraged first responders, including those 
in State and local governments to move toward digital radio technology. Numerous 
State and local governments have done so over the past decade, motivated by the 
availability of millions of dollars in Federal grant money, pending FCC narrow banding 
requirements and the need to replace aging and capacity limited analog systems. 

However, implementation of digital radio systems in the public safety environment has 
proven to be challenging, not just in Palm Beach County. There is a well documented 
history of implementation problems plaguing state and local governments throughout 
the country. The problems are not specific to any one product or manufacturer. At 
worst, such as in the case of New York State, projects were terminated altogether. In 
other cases, entities went back to their older analog system until problems with the 
digital system could be resolved. In most cases, additional network hardware and/or 
antennas had to be installed, and software upgrades and other fixes made, resulting in 
extensive delays and cost overruns. 
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While the MPSCC's implementation of OpenSky has not been without its share of 
problems as we discuss in this report, with the exception of WPB, they have gotten the 
system operational and performing to contract specifications in the remaining five 
municipalities. On balance, their results to date appear better than many other efforts 
throughout the country. The following summarizes our assessment of the operability of 
OpenSky. 

Finding (10): MPSCC HAS DEPLOYED OPENSKY IN FIVE MUNICIPALITIES 

OpenSky is currently operational in the municipalities of PBG, PB, Atlantis, Jupiter and 
Juno Beach. The following schedule shows the dates the radio system became 
operational in each municipality: 

Municipality Date Operational 

Palm Beach Gardens September 1 , 2009 

Palm Beach April 1,2010 

Atlantis December 1 5, 201 1 

Jupiter May 14, 2012 

Juno Beach May 14, 2012 

As the schedule shows, OpenSky has been operational in three of these municipalities 
from 9 months to 3 years. The system has performed to contract specifications and 
there have been no major system failures. 



The MPSCC has been effective in working with the current vendor, Harris, to overcome 
some of the earlier problems. A number of upgrades have been provided by Harris 
including, at no additional cost, a major software upgrade, an upgrade of the subscriber 
units (mobile and portable radios) from the Harris P-7200 to Harris P-7300 models and 
new noise canceling mobile unit microphones. The MPSCC has also installed three 
additional "fill in" transmission sites to address coverage issues in PBG, PB, and 
western WPB. Harris provided two of these cell site antennas at no cost. 

The MPSCC has designed OpenSky with multiple layers of redundancy. There are two 
geographically separate switching centers which house the backbone equipment for the 
system. One of the switching centers is located in a Category 5 rated building and both 
operate with three levels of power, FPL, generators and battery backup. The switching 
centers each have multiple gateway interfaces which provide for interoperability with the 
County radio system. Each switching center is served by microwave transmissions set 
up in a "ring configuration" for redundancy purposes. The primary switching center 
located in the Category 5 building is a particularly well designed site and has 5 
additional "emergency" dispatch consoles to support dispatchers from other OpenSky 
members in emergency situations. 

In April, 2012 the MPSCC submitted a request to the Florida Department of 
Management Services (DMS) for approval of their Phase 2 expansion as meeting the 
requirements of Florida's 2009 Law Enforcement Communication Plan (LECP). The 
Phase 2 expansion covers the municipalities of Atlantis, Jupiter, Juno Beach, and the 
Palm Beach County School District Police. The Phase 2 expansion includes twelve 



Page 25 



Office of Inspector General 



Audit # 201 3-A-0001 



additional transmission sites which will significantly expand the OpenSky footprint as 
well as further enhance coverage in some of the existing Phase 1 municipalities. On 
May 9, 2012, DMS approved the OpenSky Phase 2 expansion as meeting minimum 
performance standards of the 2009 LECP. 

Also in April of 2012, as MPSCC was preparing to deploy OpenSky in Jupiter and Juno 
Beach (Jupiter/Juno), they conducted an acceptance test. The test performed was an 
"outdoor" and "in-building" voice quality DAQ test, the same test that was considered a 
failure in 2009 for WPB. As part of our audit testing, we arranged to observe the 
Jupiter/Juno test. The test was conducted over three days on April 19th, 20th, and 
23rd. 

From our observation of this test, we concluded that the OpenSky system works well. 
Voice communication was generally clear and coverage met or exceeded contract 
specifications. (See details in Appendix 1). The indoor Jupiter/Juno DAQ test results 
indicate that the system performed significantly better than in the July 2009 indoor DAQ 
test that was considered a failure in WPB. 

However, we did identify one area of concern in our observation of the testing. Indoor 
test areas that failed the DAQ test, but were found to exceed the contract specification 
of 1 2dB building penetration were discarded from the test results. While this may be the 
correct testing protocol when testing OpenSky's performance against the contract 
specifications, those "failed test areas" represent a potential officer and public safety 
risk. 

It is our understanding that the same indoor DAQ testing protocol was used for all the 
MPSCC municipalities and for the second DAQ test done by a third party for WPB in 
February 201 1 . This has resulted in a number of indoor locations where OpenSky will 
not work even though it meets contract specifications. These "dead spots" should be 
documented and evaluated to determine if they pose an unacceptable level of risk and 
whether they can be resolved to mitigate officer and public safety risks. 

Recommendations: 



9) The MPSCC should establish a process to document and evaluate all failed 
indoor test locations even those that exceed contract specifications to determine 
if they pose a significant officer and public safety risk that needs to be resolved. 
Officers should be periodically reminded to document and report locations where 
radio communication failures occur so that they can be recorded, mapped and 
evaluated to determine if additional fixes are needed. 

Management Response: MPSCC 



9) In the future, documentation identifying structures that exceed contract 
specifications for building radio signal penetration will be identified as part of the 
acceptance test planning (ATP) and made part of the testing protocol. Note, as a 
standard, buildings that are known to exceed the penetration value guaranteed by 
contract are not considered a failure of the contract, nor will be in the future, 
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however it will be documented for officer safety. Other solutions implemented for 
building coverage will be determined as necessary by the particular member 
agencies. 

Finding (11): OUR RADIO SURVEY OF OFFICERS SHOWED MIXED RESULTS 
AND INDICATE POTENTIAL OFFICER AND PUBLIC SAFETY ISSUES 



As part of our audit we developed a survey form to gather feedback from officers on the 
OpenSky system in PBG, PB and Atlantis. We obtained feedback from 165 officers. 
The majority of the officers we surveyed (90%) had been using the OpenSky radios for 
more than 6 months. 

This survey provided an opportunity for officers to provide feedback on how the system 
is performing in a variety of situations that officer's encounter in carrying out their duties 
on a day to day basis. The survey instrument was designed to gauge the officers' 
opinion with respect to improvement, reliability, problem identification, problem 
frequency, problem resolution, safety, and any loss of functionality when using the 
OpenSky radio system. (See Appendix 2 for a copy of the two page survey instrument). 

The results of the survey showed that 67% agree the system is an improvement over 
the old system, 79% felt properly trained, and 65% felt the system worked properly over 
95% of the time. In terms of the Harris radio equipment, 83% rated the vehicle (mobile) 
unit as better or the same as the old system and 78% rated the portable (handheld) unit 
as better or the same as the old system. In response to a question as to whether the 
officers considered the system a total failure, 89% responded that they did not consider 
OpenSky a total failure. 

However, in response to the following statement, "There are radio problems that I 
encounter that are serious enough to adversely impact officer and public safety", 69% 
strongly agreed or agreed. Among the most commonly reported problems were dead 
spots (no signal) followed by audio quality and dropped calls. Together they represent 
82% of all the problems reported by the officers in our survey. (See Appendix 3 for a 
more complete summary of the survey results). 

Our survey instrument also provided the opportunity for the officers to make specific 
comments. The officer's survey comments were positive, negative and neutral as can 
be seen from a sample of officers' comment in response to our first "agree or disagree" 
survey statement. "The new digital radio system is an improvement over the (old) 
legacy system": 

"Life saver" 

"Voice quality and reception is greatly improved" 

"Old system had numerous dead areas in town, with the new system, these all 

nearly have been eliminated" 
"No difference noted" 

"Too many dead spots which are an officer safety issue" 
"No improvement, same problems in different areas" 
"I do not believe the system works better at all" 
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However, for our statement regarding officer safety, the comments were overwhelmingly 
negative. A sample of those comments includes: 

"Sometimes I can't understand what is said, underwater sound" 
"Dead spots/loss of signal-cannot keep up" 

"There are still some interior dead spots... Most are quickly corrected upon report" 
"Dead spots in city, radio works fine in some areas then not at all 50 ft away" 
"When I get out of the car and turn on the portable, it takes five seconds" 
"Lots of "dead spots" loss of signal in buildings such as the hospitals and some 
office buildings" 

We discussed these results with MPSCC officials. They indicated that problems being 
experienced by the officers are continually monitored and that our survey results do not 
reflect the type or extent of problems that are being reported to them. However, they 
have no formal system to document problems reported and how they are resolved. 

While we recognize that there have been ongoing improvements being made to 
OpenSky, the survey results indicate that system users believe there are problems in 
several of the municipalities that are considered fully operational and relatively problem 
free. During our observation of the more recent test in Jupiter/Juno, we did not note any 
audio quality or dropped call issues. However, as previously stated, we did note "dead 
spots" particularly indoors. 

From our observations, OpenSky performed much better in the Jupiter/Juno test than 
our officer survey results seem to indicate for PBG, PB and Atlantis. This could indicate 
the system is not performing consistently throughout the coverage area. One thing that 
is true of all public safety radio systems, whether analog or digital, is that how well they 
perform in one geographic location is no guarantee of how well they will perform in 
another. Performance is dependent to a great degree on where and how many 
transmission sites and other signal boosting devices are installed in a given area; other 
signal interference that may be occurring; and, the nature of the terrain and building 
infrastructure in that area. 

So while evidence exists to show that overall, OpenSky is performing to contract 
specifications, our officer survey results point to a potential area of risk that the MPSCC 
must address to ensure problems that could impact officer and public safety are 
promptly corrected. 

Recommendations: 



10) The MPSCC needs to focus on identifying the cause(s) and resolving the three 
major problem areas identified by users in this survey as dead spots, audio 
quality, and dropped calls. Additional system testing may be needed in these 
three municipalities to determine if additional radio infrastructure is needed to 
improve coverage. 
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11) The MPSCC needs to implement a formal incident resolution system (help 
desk) to address any concerns raised by the users of OpenSky. This system 
should follow a standard set of technical systems guideline (such as ITIL 8 ) to 
document and resolve all incidents and/or problems. This system should include 
mapping of any reported coverage problems to determine if certain locations or 
structures are chronic problem areas that need to be corrected. 

Management Response: MPSCC 



10) This is a process that we regularly perform to maintain the integrity of the 
system. We regularly survey officers and solicit their input for any areas that they 
may identify with communication problems, have fixed those problems, and will 
continue to do so. 

11) The MPSCC agrees and will develop in-house policies/procedures to achieve 
this recommendation within the next 120 days. 

Finding (12): THE QUESTION OF OPERABILITY OF OPENSKY IN WEST PALM 
BEACH REMAINS UNRESOLVED 



To date WPB has spent approximately $5,178,000 on OpenSky. However, OpenSky 
has not been deployed in WPB and acceptance testing was never completed. 

At the end of August 2009, as the MPSCC was preparing to deploy OpenSky in WPB a 
final acceptance test was conducted. The test involved 40 officers using a combination 
of Harris OpenSky radios and Motorola radios from WPB's existing Motorola system. 
This test was terminated after three days, due to numerous voice communication 
failures that were considered to pose an officer safety risk. The poor performance of 
OpenSky during this test resulted in a significant dispute between the MPSCC and 
WPB, including the WPB Technical Team (Team) who is in charge of operating WPB's 
current Motorola radio system. To date this dispute has not been resolved and the test 
that failed in August of 2009 has never been re-run. 

As previously mentioned on page 25, there have been a number improvements made to 
OpenSky since the failed August 2009 test. In particular, three additional cell sites were 
installed including one in the western part of WPB known as Ibis to resolve coverage 
problems previously identified. The software upgrades as well as the upgrade of the 
portable radios have also improved system performance. 

There have also been two additional tests performed in WPB since the failed test in 
August, 2009. In a July 201 1 report to WPB evaluating all acceptance testing 
performed on OpenSky, Harris provided documentation on a test they performed in 
January of 201 0, six months after the failed August, 2009 test. This test was a re-run of 
in-building RF Coverage/voice quality (DAQ) testing that was attempted in July of 2009 
but did not pass (outdoor testing passed). According to Harris, the re-run confirmed 



ITIL® is the International Technology Infrastructure Library standards and guidelines for technology service management support. 
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that in-building coverage met or exceeded minimum contract requirements (for 12dB 
buildings). 

Subsequent to this test, WPB contracted with an independent radio engineering firm, 
RCC Consultants, Inc. to observe another test of OpenSky. Conducted on February 
17th and 18th, 2011, this was another RF Coverage/voice quality (DAQ) testing 
procedure. Signal strength measurements were taken by three teams in 156 grids 
throughout WPB. Voice checks were also performed with dispatch and test results 
were collected by both field and dispatch teams using the Harris equipment. This test 
was conducted after the new cell tower was added near Ibis for western WPB. 

The test results showed that 151 out of 156 grids (96.79%) passed, exceeding the 
contract requirements of 95%. In reporting their overall results the RCC report stated: 

"Overall, the coverage testing results were subjectively very good in terms of the 
satisfaction of the Police officers participating in the testing" In the conclusion to 
their report RCC stated, "RCC Consultants, Inc. believes that overall, based on 
this limited test, West Palm Beach, Florida's OpenSky outdoor coverage is 
excellent with clear voice quality both in the field and at PSAP [dispatch] ". The 
RCC report also makes this statement, "Please note that in the last part of the 
testing, the three crews were checking the most challenging structures and the 
most challenging areas within those structures.. .Overwhelmingly, the majority of 
these locations passed the tests and the overall results exceeded expectations". 

The RCC test results appear to provide some credible evidence that with the 
improvements that have been made, OpenSky performed significantly better in WPB 
than the July 2009 test results indicated and met the contract requirements. However, 
in placing reliance on the RCC test it is important to point out the limitations of the test 
performed as well as other results presented in the RCC report. In describing the 
testing methodology, RCC provided the following statement which summed up the 
limitations of this test. 

"It must be noted that the testing applied in this case was not a repeat of the 
rigorous testing performed by the vendor in the past. The test that RCC 
observed represented a retest of a small citywide sample as a simplified 
verification of those previous coverage tests. Rigorous acceptance testing of a 
complex and advanced system requires specialized tools, more structured 
approach and more time. " 

While RCC's overall conclusion was that the vendor passed this test, their report 
included a section that discussed what they labeled as "trouble spots". They listed 
thirteen buildings and/or specific locations within buildings where there were heavy RF 
penetration losses exceeding 12dB. The list included three hospitals, the basement of 
the Police Station and a high school. RCC stated that this list represents only those 
buildings that were tested. This would seem to indicate that a full test might identify 
additional buildings with RF penetration losses exceeding 12dB. In their report, RCC 
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noted the Palm Beach County Fire Code 9 requires the owner's of buildings that fail the 
RF penetration test install bi-directional amplifiers (BDA) to resolve the signal strength 
problem. 

Based on all the test results we have reviewed, we have two conclusions regarding the 
operability of OpenSky in WPB. First, a more rigorous test of the current OpenSky 
configuration is needed in WPB to validate the results of the more limited test that RCC 
observed. At a minimum this would need to include a test of signal strength (RF 
coverage testing), voice quality (DAQ) and system reliability (14 day Reliability Test). 
Second, in order for OpenSky to meet the needs of WPB and mitigate officer and public 
safety concerns, the system will have to perform better than current contract 
requirements. Performance only meeting current contract specifications, specifically 
penetration of buildings up to a loss factor of 12dB, would leave a number of buildings 
with no radio coverage as noted in the RCC test. Also, Harris in their July 2010 report 
to WPB that evaluated acceptance testing, invited WPB to work with them to identify 
areas requiring additional in-building penetration, an acknowledgement that the current 
contract requirement for 1 2dB penetration is not sufficient for WPB. 

Subsequent to the RCC 
developed a new 
OpenSky implementation 
plan (Figure 2). The 
implementation was 
scheduled to begin on 
May 1, 2011 and be 
completed on September 
1, 2011. However, 
because of the issues and 
concerns that had been 
raised by WPB, the plan 
was never implemented. 



We were provided 
documentation from the 
MPSCC regarding a 
solution Harris had 
proposed for WPB that 
would require installation 
of two additional transmission sites. The estimated cost for this is $440,000. We are 
also aware that Harris has been discussing with WPB another option to provide WPB 
with a Harris P-25 IP digital radio system. We have not seen the cost estimate on that 
proposal. However, it is not clear whether acquiring a separate Harris P-25 IP system 
without also addressing the need for more signal strength by installing additional 
transmission sites will provide WPB with an adequate solution. The MPSCC and WPB 



9 Palm Beach County Local Amendments to the Florida Fire Prevention Code, 2008 Edition: (10.23) In all new and existing buildings 
and structures, minimum radio signal strength of -102.0 dBm (1.78 micro volts) in the frequency band of 806-824 / 851-869 MHz 
shall be maintained. Where this signal strength cannot be achieved, an 800 MHz bi-directional amplified system shall be installed to 
meet minimum radio signal strength required for effective emergency communications 
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need to work together to ensure that any option involving OpenSky or another Harris 
product would meet the needs of both the MPSCC and WPB. 

Recommendations: 



12) Before deciding whether to proceed with deploying OpenSky in WPB, the 
MPSCC and WPB need to perform a full DAQ test with sufficient test points to 
identify all buildings that have a loss fact greater than 12dB and determine what 
level of signal strength is needed to penetrate those denser buildings. 

13) If the MPSCC and WPB decide to move forward with OpenSky or another 
Harris system, a separate contract with Harris should be executed that includes 
specifications that meet WPB's more challenging infrastructure. The contract 
should also provide adequate protections for the MPSCC and WPB if system 
performance does not meet contract specifications. 

Management Response: MPSCC 



12) West Palm Beach should perform a full DAQ test as suggested by the 
Inspector General. 

13) Should the City of West Palm Beach move forward with OpenSky or another 
Harris system an amendment to the Interlocal agreement must be completed with 
the MPSCC and a contract executed with Harris. 

Management Response: WPB 



12) We concur. 

13) We concur. 
OIG Comment 



Specific actions needed to address these recommendations will be contingent 
upon decisions made by WPB and/or the MPSCC on whether to deploy OpenSky 
or pursue an alternative solution. We will follow up on any actions needed to 
address these recommendations once that decision has been made. 
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Finding (13): GOING FORWARD THE CITY OF WEST PALM BEACH CAN CHOOSE 
AMONG SEVERAL OPTIONS FOR A PUBLIC SAFETY RADIO SYSTEM 



The city of West Palm Beach has a number of options for meeting their public safety 
radio needs. As part of our review we did an analysis of four options that could be 
implemented without a lengthy system development and acquisition process. Our 
analysis included an estimated five year cost for each option. However, as we discuss, 
the two County options have potential future costs that cannot be fully identified without 
a test of current system coverage. The four options include: 

1 . Joining the County's Motorola SmartZone 3.0 as a Direct Connect customer, 

2. Joining the County's Motorola SmartZone 3.0 as a Hub Connect customer, 

3. Continuing on WPB's current Motorola SmartNet system, or 

4. Remaining with MPSCC on the OpenSky system. 

Financial data related to the first two options was based on a presentation made by 
Palm Beach County's Facilities Development and Operations Department to the West 
Palm Beach City Commission. The comparison of these four options was based on the 
following assumptions: 

(1 ) 557 radios in total for WPB Police Department; 

(2) 18 talk groups, SmartX and MTC Controller used for the County's Hub 
Connection option; 

(3) 9 frequencies transferred to the County for the County's Direct Connect Option; 

(4) 7 dispatch consoles used for either the County's Direct Connect or Hub Connect; 
and 

(5) County's current resolution (R-2002-01 92) for $1 2.50 reimbursement. 

We did not include WPB's current investment of over $5.1 million in OpenSky in our five 
year cost analysis since this is a sunk cost and could not be recovered under any of the 
options. The following is a brief discussion of each of these options: 

Motorola Direct Connect: Five Year Gross Cost $2,706,268 

Under this option, WPB would continue to operate an analog radio system. However 
their five year gross cost could be reduced by as much as $1 ,1 70,000 from a County 
"frequency credit" if they transferred all 9 of their frequencies to the County. WPB 
could also offset up to another $464,986 in five year costs if they received their full 
share of $12.50 money under the current County resolution. The "frequency credit" 
and $12.50 offset would result in a five year net cost of $1 ,071 ,282 , which would 
make this the least costly of the four options based on known costs. 

However, this option could result in additional future costs. The County will 
eventually convert to a digital radio system and at that time WPB would need to 
upgrade or replace their current Motorola radios. Based on information provided to 
us by WPB radio support staff, sufficient radio inventory exists that can be upgraded 
to P-25 standards and this would cost approximately $120,000. The County's 
timeline for converting the County analog system to a digital radio system has not 
been firmly established. However, the County stated that the current 
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renewal/replacement purchase for their aging Motorola backbone equipment would 
allow them to operate as an analog system for six to eight more years. 

This option could result in other infrastructure costs. The level of RF signal strength 
coverage the County system could provide to WPB as their public safety radio 
system provider has never been fully tested. Considering RF signal strength and 
building penetration issues already identified in WPB by RCC in the DAQ test of 
OpenSky, an indoor and outdoor full DAQ test of the County system in WPB would 
need to be performed. If additional RF signal strength is needed, WPB would incur 
additional infrastructure costs which could include antenna changes and/or required 
in-building BDA's. However, those costs are unknown until a full city-wide DAQ test 
is performed. 

Motorola Hub Connect: Five Year Gross Cost $4,437,006 

Under this option, WPB would continue to operate an analog radio system. The 
County HUB Connection is the most costly option over the next five years. The five 
year gross cost less the "12.50 offset" of $464,986 would be a five year net cost of 
$3,972,020. Like the County Direct Connect option, there would be additional future 
costs of approximately $120,000 to upgrade their current Motorola radios to the P25 
digital standard once the County converts to a digital radio system. Also, the 
eventual conversion to a digital radio system could involve additional infrastructure for 
WPB to ensure adequate RF coverage for transmission of a digital signal. Without 
performing the required full DAQ testing, those costs cannot not be reliably 
determined. 

Current Motorola SmartNet System: Five Year Gross Cost $3,207,915 

Under WPB's current Motorola analog 800 MHz SmartNet radio system, WPB spends 
approximately $641 ,583 every year for maintenance/operations. Their five year gross 
cost less the "12.50 offset" of $464,986 would be a five year net cost of $2,742,929 . 
However, Motorola no longer provides the maintenance service since the system has 
reached the end of its life cycle. Over time, maintenance and support could become 
more difficult and more costly. In their September 2010 internal report on OpenSky, 
the WPB Technical Team that supports the SmartNet system indicated that at that 
time, they had three years of spare parts to maintain their current system at existing 
levels. Based on this, staying with the Current Motorola SmartNet system is not a 
viable option beyond 2013 and could put WPB public safety radio communications at 
risk if hardware failures occur that cannot be immediately fixed. 

OpenSky: Five Year Gross Cost $2,879,082 

Under this option WPB would convert to a digital radio system. Two one-time costs 
comprise over 70% of OpenSky's five year gross costs. This includes the additional 
costs to purchase the remaining 419 Harris radios needed, at a cost of $1,672,805. 
The other one-time cost is the estimated $440,000 needed to install two additional 
transmission sites to address in-building RF penetration issues identified in WPB. 
The remainder of WPB's five year gross cost for OpenSky consists of annual dues to 
the MPSCC currently budgeted at just over $153,000 per year. 
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Under the current County resolution R-2002-0192, WPB's costs for OpenSky would 
not be eligible for $12.50 offset. Therefore, the five year net cost would remain at 
$2,879,082 . However, we are aware that based on the Palm Beach County Board of 
County Commissioners workshop held on August 21, 2012, County Administration is 
preparing a proposed County resolution that would allow municipalities on the 
OpenSky system to receive reimbursement for eligible expenses equivalent to their 
share of $12.50 money. If this resolution passes, WPB could receive up to the same 
$464,986 offset that we factored into the other three options. This would result in a 
five year net cost of $2,414,096. Also, since our estimate includes the cost to 
purchase the additional Harris digital radios as well as additional infrastructure to 
address in-building RF penetration, there are no other unknown future costs to 
consider with this option. 

As discussed earlier in this report, it is our understanding WPB is also in discussions 
with Harris Corporation for a fifth option that would include purchasing the Harris P-25 IP 
digital public safety radio system. We have not seen a cost estimate for this option. 

Recommendations: 



14) WPB should carefully consider each of their options. To assist in making the 
best decision, WPB should consider using an independent radio system 
consultant to thoroughly evaluate the City's needs, including their unique and 
more challenging infrastructure needs. 

15) WPB should have their radio Technical Team assess the timeframe left for 
replacing their current aging system so that can be factored into any decision on 
moving forward with any of their available options. 

Management Response: MPSCC 



14) West Palm Beach has hired a consultant to evaluate their options. 

15) MPSCC concurs. 
Management Response: WPB 



14) We concur. An independent consultant was hired in September 2012. 

15) We concur. 

Management Response: Palm Beach County 



14) &15) The County agrees with Recommendations 14 and 15 but disagrees with 
some of the direct statements made/financial and technical conclusions 
drawn and/or implied by the findings leading up to those recommendations. 
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FUTURE PLANS FOR MAINTENANCE AND SUPPORT OF OPENSKY 



Finding (14): PLANS FOR SYSTEM SUPPORT, MAINTENANCE AND FUTURE 
PLATFORM MIGRATION ARE BEING ADDRESSED BY THE MPSCC 



Support and Maintenance 

The contract with M/A-COM (Harris) for the purchase of the OpenSky public safety radio 
system includes language that requires Harris to provide, "full availability of all parts, 
components or comparable parts and service for a period of seven (7) years on all 
Seller manufactured infrastructure equipment and for five (5) years on all subscriber 
equipment from the last date of manufacture." 10 

A system management agreement is in place between the MPSCC and Citation 
Communications which includes provision of services such as: 

• Monitor and troubleshoots radio, microwave, and other wireless communications 
devices. 

• Providing input to determine communication network hardware and software 
equipment needs including installation of equipment. 

• Maintaining backup and restores capabilities of all communications, security, 
and software files including policies, and security for the communications 
network. 

• Testing and evaluates new telecommunications hardware and software. 

• Maintaining inventory of communications computer hardware and components. 

• Maintaining a system for evaluation and reporting communications network 
performance, usage and twenty-four hour emergency contacts. 

• Maintaining database of FCC licenses and providing assistance with FCC 
license issues. 

• Annual radio site inspections and testing. 

• Developing and maintaining a database of all radios used on the system. 
Migration 

On August 2, 2010 Harris Corporation announced the next generation of the OpenSky 
radio system, OpenSky2. OpenSky2 utilizes the same radios, dispatch consoles, and 
backbone equipment utilized with existing OpenSky systems. The OpenSky2 system 
"incorporates significant technical and service advancements into the OpenSky 
platform." 

In keeping with the technical changes in public safety radio communications, the 
MPSCC plans to migrate the current OpenSky system to the newer OpenSky2 platform. 



10 Citation Communications has stated that the existing OpenSky Harris P7300 radios and the C3 Maestro dispatch 
consoles being utilized in the system are currently being manufactured by Harris Corporation. 
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Funding Future System Replacement 

As of September 30, 2011, the MPSCC has $448,821 in its business reserve bank 
account. The account was established in June 2006. From our review of the financial 
records, we noted the account is essentially the MPSCC's savings account and together 
with a separate checking account has been used to deposit revenue and cover 
expenses. It has not been established as a separate restricted reserve account for 
funding future system replacement. Also, the reserve amount is not based on any 
analysis of the future cost to replace OpenSky at the end of its useful life. As such, the 
account may not have an adequate balance to fund the eventual system replacement. 

Recommendations: 



16) The MPSCC should begin planning for system replacement and consider 
collecting additional funds from members to establish a separate reserve account 
to adequately fund future system replacement costs. 

Management Responses: MPSCC 



16) Within the next year the MPSCC will develop a program to sufficiently address 
the recommendation suggested by the Office of Inspector General. 



ADMINISTRATIVE AND ACCOUNTING PROCEDURES AND CONTROLS 



Since MPSCC's inception, it has received funding of $7.1 million from various sources: 
grants, loans, membership dues, project implementation funds, deposits, and interest 
revenue. Identified funding from FY 2002 to December 2005 (before the contract was 
executed with M/A-COM) was $71 1 ,091 and expenditures were $75,395. From January 
2006 to the end of fiscal year 2011, MPSCC received $6.43 million and spent $6.08 
million. The available balance as of September 30, 201 1 was $670,761 . 

As part of our review, we tested expenditures totaling $5.1 million to verify funds were 
properly spent and accounted for. Our sample did not identify any improper 
expenditures. 

However, in performing the audit work, we noted that certain MPSCC's accounting and 
administrative procedures and processes were not well developed. The MPSCC has no 
full-time salaried staff, and accounting was performed as a collateral duty by police 
department personnel from member municipalities. Prior to 2006 accounting duties 
were performed by the former Chief of Police of Boynton Beach and since January 2006 
by the Special Projects/Telecommunications Officer and the Fiscal Coordinator, Palm 
Beach Gardens Police Department. Various administrative duties have also been 
carried out by municipal staff as well as one of MPSCC's contractors. We question 
whether the current MPSCC staffing arrangement is sustainable and we have specific 
recommendations to address this in discussing the following findings related to 
accounting and administrative weaknesses. 
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Finding (15): MPSCC LACKS A FORMAL ACCOUNTING SYSTEM AND SINCE 
INCEPTION HAS NOT HAD COMPLETE AND AUDITED FINANCIAL STATEMENTS 

During the past ten years, MPSCC has received millions of dollars from its members, 
from Federal grants, and from loans. In order to account for all the revenue and 
expenditure transactions, we obtained the bank statements from 2006 to 201 1 and the 
detailed transaction records from MPSCC. We were unable to obtain any bank 
statements prior to January 2006. Since there was no chart of accounts or account 
codes, the transaction records were prepared using Microsoft Excel by a PBG's Police 
Department Fiscal Coordinator, and no general ledger and sub-ledgers were prepared. 
As a result, we were unable to trace activities that occurred prior to January 2006 to 
bank statements. 

We also noted that MPSCC has not prepared financial statements on an accrual basis 
from FY 2000 to FY 201 1 that could then be audited by an independent auditor. 
According to the MPSCC's Inter-local Agreement Section 8.K, which was signed in 
December 1999, an annual audit of financial records should be conducted by an outside 
auditing firm at the end of each fiscal year. However, the governing board had not 
arranged any annual audits even though the consortium members had agreed to 
require it. 

Without a more formal accounting system and without any part time or full time salaried 
staff with the appropriate accounting expertise, the MPSCC could find it a challenge to 
prepare complete and accurate financial statements that could then be audited. The 
only financial statement schedules we found are in the MPSCC's informational tax 
returns prepared by Caler, Donten, Levine et al, P.A. which contain a Statement of 
Revenue, Expenses and Changes In Net Assets or Fund Balances and Balance Sheet 
prepared on the cash basis of accounting. As a non-profit government corporation the 
MPSCC's annual financial statements would need to be prepared on an accrual basis. 

The MPSCC has already taken action to address a number of these accounting issues. 
We were notified by MPSCC's Executive Director that the annual audit had been 
budgeted for FY 2013 and an external auditor will be hired to perform the MPSCC's 
annual financial audit. MPSCC also intends to hire accounting personnel to maintain 
financial records and prepare the financial statements on an accrual basis. 

Recommendations: 



17) We recommend that MPSCC arrange for an annual financial audit. The annual 
audit is an essential component to show MPSCC's accountability for 
contributions and expenditures according to the mission of the organization. The 
annual audit should be conducted by an independent audit firm and audit fees 
should be budgeted accordingly. 
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18) The Board of Directors should also consider hiring professional accounting 
personnel or a competent third-party to manage MPSCC's accounting activities 
and prepare the annual financial statements. Structured and timely accounting 
should be established for the long-run to handle the substantial funds from/to 
various sources. 

Management Responses: MPSCC 



17) The current budget (2012/2013) established by the MPSCC contains funding 
for such an audit to be conducted. 

18) Currently a scope of work is being developed, after which, through a proper 
procurement bid, personnel will be hired based on the job description and scope 
of work developed. Projected timeline is 120 to 180 days. 

Finding (16): MPSCC LACKED CERTAIN KEY FINANCIAL CONTROL POLICIES 
AND PROCEDURES 



During our review of MPSCC's internal control policies and procedures, we noted that 
the MPSCC only had purchasing guidelines & accounts payable procedures, and a cost 
allocation & revenue policy. 

No annual budget policy : We noted that there was no budget policy and no budget-to- 
actual comparison had ever been made at the end of each fiscal year. Therefore, the 
actual expenditures exceeded the budgeted amount for three consecutive fiscal years - 
2008, 2009, and 2010. When the actual expenditures exceeded the budgeted amount, 
the reserve fund was applied without formal approval. 

Since there was no budget policy, each line item on the annual budget was not clearly 
defined as to purpose and use. For example, one of the budget line items "Harris 
software license" included upgrades not only for software but also for hardware (radios, 
consoles, and infrastructure equipment). 

During our review of MPSCC's financial records, we noted that the MPSCC spent a total 
of $101,738 ($31,200 in federal grant money and $70,538 in municipality contributions) 
from FY 2003 to FY 201 1 for legal services. The actual charges for attorney services 
were $28,352 for FY 2006 and $24,187 for FY 2011 while the budgeted amount was 
$10,000 for FY 2006 and $15,000 for FY 2011, resulting in an unapproved variance of 
$27,539. 

Lack of procedures for tracking fixed asset : We noted that the fixed assets did not have 
any tags in place and there was no tracking system for the location of the fixed assets. 
However, the security around the fixed assets which remained with the MPSCC 
appeared adequate. 

No travel and reimbursement policy : In the early years of the MPSCC, members 
traveled out-of-state for project training and meetings. It was our understanding that a 
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related travel policy or expenditure reimbursement policy related to members' travel did 
not exist. 

No document retention policy : We noted that a majority of documents of the MPSCC 
dated from FY 2000 to FY 2005 were stored with the Criminal Justice Commission. 
However, the documents were not archived in an organized manner. From FY 2006 and 
on, MPSCC began to retain its own documents in appropriate categories. However, no 
document retention policy has been established. 

Instance of a lack of segregation of duties : During the invoice-payment process, the 
Fiscal Coordinator performed two duties for the MPSCC - recording the transactions 
and verifying the delivery of goods. 

After bringing these matters to their attention MPSCC has begun taking action to correct 
these weaknesses. Actions being taken include: 

1. MPSCC planned to keep more than minimum required amount in a reserve 
account and a chart of accounts will be established; 

2. MPSCC will set up a monthly tracking form for all of the line-item accounts and 
the Board of Directors will review it at the MPSCC's monthly Board meeting. 
This will minimize the chance of over-spending. 

3. MPSCC will establish separate budget line items for software upgrade and 
hardware (radios, consoles & infrastructure equipment) upgrades; 

4. For all other policies and procedures mentioned above, MPSCC started to 
establish them accordingly. For example, MPSCC began building a database for 
the fixed assets and was waiting for asset tag orders. The database will list the 
entire infrastructure owned by MPSCC. 

Recommendations: 



19) There should be a Board agreement/policy concerning budget and budget-to- 
actual reviews, and the purpose and the use of operating reserves. This policy 
should define and set goals for reserve funds, describe authorization for use of 
reserves and outline requirements for reporting and monitoring. 

20) The MPSCC needs to establish a policy for fixed assets including the use of 
fixed asset tags. 

21) MPSCC needs to establish travel policies including types and amounts of 
allowable expenses. 

22) MPSCC should establish a document retention policy. 

23) The organization should have policies established for segregation of duties of 
key transactions. The person who records the transactions should not be the 
person who also verifies the delivery of goods. The duties of authorization, 
recording, custody and reconciliation should be segregated. 
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24) The MPSCC should consider hiring administrative staff to ensure that all of 
the additional policies, procedures and processes that need to be put in place get 
timely established and are consistently maintained and carried out going forward. 

Management Response: MPSCC 



19) The board will develop a policy documenting procedures for the above 
recommendation within the next 120 days. 

20) The board will develop a policy documenting procedures for the above 
recommendation within the next 120 days. The MPSCC, at the suggestion of 
Inspector General, already started the fixed asset tagging process and will be 
completed within the next 90 days. 

21) Policy will be established within the next 90 days. 

22) Policy will be established within the next 90 days. 

23) The formal policy will be developed within the next 120 days; however 
implementation of that policy is currently being utilized to achieve the best 
segregation of duties possible. 

24) As mentioned in recommendation 18, we are currently preparing the scope of 
work (under development). 

Finding (17): THE MPSCC DID NOT ALWAYS COMPLY WITH EXISTING 
ACCOUNTS PAYABLE AND PURCHASING PROCEDURES 



We selected a sample of ninety expenditures to test the accounts payable procedures. 
Seventy-four expenditures (82.% of the testing population) complied with the accounts 
payable procedures. However, sixteen payments (18%) did not comply. Five payments 
totaling $32,656 showed only one signature on each check to authorize the payment, 
while the MPSCC's accounts payable procedures required two authorized signatures. 
The remaining eleven payments totaling $55,647 were each over $2,500 and per 
accounts payable procedures, required Board approval. Those were necessary 
expenditures. However, we were unable to obtain any supporting documents showing 
Board approval. 

We also found that the MPSCC tax preparer was recommended directly by MPSCC 
personnel and their services were purchased without first obtaining three quotes. Per 
MPSCC's Procurement policy, expenditures under $10,000 require three quotes. From 
2005 to 201 the tax preparer fees ranged from $71 5 to $1 ,775 per year. 
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Recommendations: 



25) The governing board of MPSCC should ensure that all the activities are in 
compliance with internal control standards. 

Management Response: MPSCC 



25) The MPSCC agrees. Quarterly meetings are already being planned and will be 
established for compliance review, within the next 90 days. 



ACHIEVING THE GOAL OF INTEROPERABLE COUNTYWIDE RADIO 

COMMUNICATION 



Finding (18): THERE IS A NEED TO REESTABLISH A COUNTYWIDE PUBLIC 
SAFETY RADIO COMMUNICATIONS COMMITTEE 



Going back to the early 1990's the County and its various law enforcement entities 
recognized that with the number of separate law enforcement/public safety agencies 
operating in the county and with many having their own communications systems, they 
were unable to effectively communicate with each other. A number of efforts were 
undertaken to begin to address this situation. However, in 1994, after extensive 
research, the County Communications Committee advised that state of the art 
technology was not sufficient to provide a large enough network to accommodate the 
needs of the County and the needs of the municipalities, other than on an emergency 
basis only. All parties were disappointed, frustrated and concerned. Extensive friction 
existed among the cities and the county entities. The Criminal Justice Commission 
resolved this dissension by establishing the Countywide Public Safety Communications 
Committee (CPSCC). The purpose of the CPSCC was to focus on law enforcement 
and public safety communication planning for municipalities throughout the county. 

Several years later, in 1999 the County established, through a County resolution, the 
Communications and Systems Operations Policy Advisory Committee (CSOPAC). 
CSOPAC's mission was to develop a countywide technical, operational and financial 
plan for implementation of an interoperable communications system for both public 
safety and general government agencies. CSOPAC membership included both County, 
municipal and other public safety agency officials. It was provisioned to sunset in two 
years. 

The CPSCC evolved into the MPSCC which went on to acquire, implement and manage 
the M/A-COM (Harris) OpenSky system. The CSOPAC sunset in 2001 and the during 
that time the County went on to acquire, implement and manage the Countywide 800 
MHz Motorola SmartZone system. 

OpenSky while serving far fewer municipalities than originally envisioned still represents 
a significant investment of both time and money in establishing an interoperable 
municipal public safety radio communication system. With the Palm Beach County 
School District Police currently implementing OpenSky, the system will have a more 
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extensive countywide footprint. Also, the MPSCC recently received approval from the 
Florida Department of Management Services for OpenSky's Phase 2 configuration as 
meeting the minimum requirements of the State Law Enforcement Communication Plan. 

Considering that five municipalities are currently operating on OpenSky, the Palm 
Beach County School District police are implementing OpenSky, and Palm Beach 
County is moving toward a major upgrade of the County's Motorola system; it may be 
appropriate to reestablish a committee like the CPSCC or CSOPAC. Such a committee 
could help ensure more cohesive planning and coordination that achieves appropriate 
levels of interoperability, while providing the various public safety entities flexibility in 
choosing among systems and technologies available now and in the future. 

Recommendations: 



26) Establish a countywide public safety radio communications committee to 
ensure coordination and compatibility between all organizations involved in 
implementation, support, and/or use of public safety radios. 

Management Response: MPSCC 



26) The MPSCC believes that previously established committees, that have since 
been disbanded, allowed for dialogue and views for public safety 
communications to at least be shared for discussion and for familiarization of 
other systems. Hopefully a new committee can be formed so that proposed radio 
planning and philosophies for the future can be shared. We wholeheartedly 
support the development of this committee. 

Management Response: Palm Beach County 



26) There are several existing operational planning committees already in place 
which have, in whole or in part, overlapping memberships and/or 
communications system missions. The establishment of an additional planning 
committee will dilute the existing committees' attendance/success. 

However, information sharing is warranted so that each entity can plan its future. 
To that end, the County would agree to schedule, host and staff an annual 
Public Safety Radio System Communications Information Sharing Workshop with 
the administrator, police chief and fire chief from each agency being invited to 
attend. 
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QUESTIONED COST 



Questioned Cost: $253,863 



ATTACHMENTS 



Attachment 1 - West Palm Beach Management Response 
Attachment 2 - Palm Beach County Management Response 
Attachment 3 - MPSCC Management Response 
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APPENDIX 1 - Jupiter/Juno Beach DAQ Test 



The first test day, April 19, 
2012, was focused on testing 
radio voice quality outdoors 
and the second day and third 
days were focused on indoor 
testing. A Delivered Audio 
Quality (DAQ) score of 3.4 or 
above was required for 
passing the test. Four 
personnel from Office of 
Inspector General (OIG) 
audit team observed the 
three-day radio testing. 



uenvereu auqio 
Quality 


Subjective Performance Description 


DAQ 5.0 


Speech easily understood. 


DAQ 4.5 


Speech easily understood. Infrequent Noise/Distortion. 


DAQ 4.0 


Speech easily understood. Occasional Noise/Distortion. 


DAQ 3.4 


Speech understandable with repetition only rarely required. Some 
Noise/Distortion. 


DAQ 3.0 


Speech understandable with slight effort, Occasional repetition 
required due to Noise/Distortion. 


DAQ 2.0 


Understandable with considerable effort. Frequent repetition due 
to Noise/Distortion. 


DAQ 1.0 


Unusable, speech present but unreadable. 



According to the testing map grids, participants were divided into three teams. Team 1 
was in charge of the orange grids, team 2 the pink grids and team 3 blue grids. Each 
team included three 
persons- one officer from 
Jupiter Police Department, 
one person from Citation 
Communications and one 
auditor from OIG audit 
team. One OIG auditor was 
stationed each day at the 
Palm Beach Gardens Police 
Department Dispatch 
Center. OIG team 
alternated daily to a 
different team and the 
Dispatch Center for 
observation. A total of 142 
grids were tested. We 
observed the following 
results: 



Outdoor Test (April 19th) 
During the outdoor testing, the team would pass through each grid. The voice quality 
test was conducted at a randomly selected location along the driver route within each 
grid using the portable radio. Team members would exit the vehicle, call dispatch and 
record the clarity of the call, DAQ 1 .0 to 5.0. We observed the calls which had clear and 
intelligible communication. 
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APPENDIX 1 (Continued) 



Indoor Test (April 20th, 23rd) 

During in-building testing, test calls were placed from the portable radio to the 
dispatcher from locations within a building. If a building failed the voice quality test at 
any location, then additional tests were made at other locations within the building to 
determine if the loss characteristics of failed test locations exceed MPSCC's specified 
12dB. Testing was completed in apartment buildings, homes, schools, a medical 
center, banks, various stores, restaurants and bars. 

A few locations exhibited failed loss characteristics: 

• Beacon Cove elementary school lobby area; 

• Two bank locations within the safety deposit area (Predicted dead spots 
exceeded 12dB), 

• A Condominium in Jupiter, 300 Ocean Trail (Parking garage only exceeded 
12dB), 

• A Condominium in Jupiter, 176 Helios Drive (East stairwell exceeded 12dB), 

• Juno Beach Resort (Second floor west stairwell exceeded 1 2dB), 

• Jupiter Medical Center first floor beside elevator (This location exceeded 1 2dB); 

• The Bear's Club inside one of the townhomes (The Bear's Club communications 
was determined to be an antenna adjustment issue), 

• Wal-Mart - the center of store (one location exceeded 1 2dB); 

• A Condominium in Juno Beach. 

In the Acceptance Test Procedures for indoor DAQ, the 95% "reliability acceptance 
criteria" only includes locations (test points) that are within the 12dB requirements of the 
contract specifications. Test Points that exceed the contracted 12dB penetration 
requirement and score less than 3.4 are "discarded" and not included in the calculation 
of reliability acceptance. Therefore, it should be noted that the acceptance rate 
calculation for the indoor DAQ test does not include the discarded failed test points . 
Outdoor DAQ tests include all test points. 

Dispatch 

We also observed the communication at the Palm Beach Gardens Police Department 
Dispatch Center. We heard clear and intelligible communication except on the above 
listed areas. The Palm Beach Gardens dispatcher provided us with a excel spreadsheet 
for the three days of testing. Each row of the spreadsheet contains the Team Number, 
Grid Number, Address (or location description), and a "pass or fail" indicator for each 
test point 
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APPENDIX 2 - Survey Questioneer 



OpenSky Radio System Survey code a 

SECTION ONE: 

KEY for Answering: Circle your answer and if needed, add a comment. 

SA - Strongly Agree 

A ■ Agree 

D - Disagree 

SD - Strongly Disagree 

1. The new digital radio system is an improvement over the (old) legacy system. 

SA - A - D - SD Optional Comment: 



2. The radio works properly over 95% of the time. 

SA - A - D - SD Optional Comment: 



3. The radio system problems that I report are resolved in a timely manner. 

SA - A - D - SD Optional Comment: 



4. There are radio problems that I encounter that are serious enough to adversely impact officer or 
citizen safety. 

SA - A - D - SD Optional Comment: 



5. Training has provided me with sufficient expertise to best utilize the new radio equipment. 

SA - A - D - SD Optional Comment: 



6. This new system requires me to carry a second communications device in order to be assured that I 
will have communications at all times. 

SA - A - D - SD Optional Comment: 



7. This new system has the "interoperability" that allows me to have radio communications with other 
public safety departments outside of my municipality that are not on OpenSky. 

SA - A - D - SD Optional Comment: 



8. This new system is a total failure. 

SA - A - D - SD Optional Comment 



Survey V9 Page 1 of 2 
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APPENDIX 2 (Continued) 



OpenSky Radio System Survey code a 



SECTION TWO: 



KEY for Answering: Check all that apply. 



1. What kind of problems has your radio equipment experienced, how often and where? 

Kind / How Often / Where 



(a) Dropped calls / Daily, Weekly, or Occasionally / Indoors Outdoors In Vehicle 

(b) Audio quality / Daily, Weekly, or Occasionally / Indoors Outdoors In Vehicle 

(c) Mic Sensitivity / Daily, Weekly, or Occasionally / Indoors Outdoors In Vehicle 

(d) Dead spots / Daily, Weekly, or Occasionally / Indoors Outdoors In Vehicle 

(e) Other; 

(f) None (Kmc! of Problem, Hew Often, and Where) 

KEY for Answering: Check one and if needed, add a comment. 

2. How long have you been using the new digital radio system? 

Optional Comment: 



(a) Less than one month 

(b) More than one month 

(c) More than three months 

(d) More than six months 

3. How would you rate the new digital Vehicle unit as compared to the (old) legacy system? 

Optional Comment: 



(a) Better 

(b) Worse 

(c) About the same 



4. How would you rate the new digital Portable unit as compared to the (old) legacy system? 

Optional Comment: 



(a) Better 
\ (b) Worse 
(c) About the same 



5. Are there any important feature(s) or functionality that you feel you have lost with the switch to the new 
digital system? 



(a) No 

(b) Yes 



If Yes, List/Describe lost feature(s): 



SECTION THREE: 

Add any commentfs) you think may be important in making the OpenSky project successful 
or any "show stopper" that can restrict OpenSky from being successful 



Survey V9 



Page 2 of 2 
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APPENDIX 3 - Summary Survey Results 



Score Sheet for OpenSky Radio System Survey 



SURVEY COMBINED Total number of Forms = 165 


Section One: 

Statements 










SA 


A 


D 


SD 


I 


SA+A 


D+SD 




1 




32 


73 


41 


12 


158 










20.3% 


46.2% 


25.9% 


7.6% 




66.5% 


33.5% 


2 




28 


76 


47 


9 


160 










17.5% 


47.5% 


29.4% 


5.6% 




65.0% 


35.0% 


3 




25 


71 


31 


14 


141 










17.7% 


50.4% 


22.0% 


9.9% 




68.1% 


31.9% 


4 




56 


55 


37 


13 


161 










34.8% 


34.2% 


23.0% 


8.1% 




68.9% 


31.1% 


5 




28 


98 


26 


8 


160 










17.5% 


61.3% 


16.3% 


5.0% 




78.8% 


21.3% 


6 




35 


51 


46 


27 


159 










22.0% 


32.1% 


28.9% 


17.0% 




54.1% 


45.9% 


7 




27 


80 


24 


16 


147 










18.4% 


54.4% 


16.3% 


10.9% 




72.8% 


27.2% 


8 




3 


14 


82 


56 


155 










1.9% 


9.0% 


52.9% 


36.1% 




11.0% 


89.0% 






Section Two: 

Questions 


1 




DroDDe 
d Calls 


Audio 
Qualitv 


Mic 
Sensitivit 


Dead 
Spots 


Other 


None 


I 








60 


90 


35 


118 


11 


12 


326 






18.4% 


27.6% 


10.7% 


36.2% 


3.4% 


3.7% 




2 




< One 
Month 


> One 
Month 


> Three 
Months 


> Six 
Month 















10 


6 


144 






160 






0.0% 


6.3% 


3.8% 


90.0% 








3 




Better 


Worse 


Same 














75 


25 


48 








148 






50.7% 


16.9% 


32.4% 










4 




Better 


Worse 


Same 














72 


36 


52 








160 






45.0% 


22.5% 


32.5% 










5 




No 


Yes 
















120 


36 










156 






76.9% 


23.1% 
















Section Three: 

This is a "Comment Only" section. 



Excellent 
Good 
Poor 
Bad 

Info Only 



Statement: 

The new digital radio system is an improvement over the (old) 
legacy system. 

The radio works properly over 95% of the time. 



The radio system problems that I report are resolved in a 
timely manner. 

There are radio problems that I encounter that are serious 
enough to adversely impact officer or citizen safety. 

Training has provided me with sufficient expertise to best 
utilize the new radio equipment. 

This new system requires me to carry a second 
communications device in order to be assured that I will have 
communications at all times. 

This new system has the "interoperability" that allows me to 
have radio communications with other public safety 
departments outside of my municipality. 
This new system is a total failure. 



Question: 

What kind of problems has your radio equipment 
experienced, how often and where? 



How long have you been using the new digital radio system? 



How would you rate the new digital Vehicle unit as compared 
to the (old) legacy system? 



How would you rate the new digital Portable unit as compared 
to the (old) legacy system? 



Are there any important feature(s) or functionality that you 
feel you have lost with the switch to the new digital system? 
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401 Clematis Street (33401) 
West Palm Beach, FL 33402 
Tefephone: 561-822-1400 



e-mail: emitchell@wpl> .org 



City Administrator 



Fax: 561-822-1424 



P.O.Box 3366 



Ed Mitchell 



The Capita! City of the Palm Beaches 



November 19,2012 

Dennis Sch in del , Director of Audits 

Office of Inspector General, Palm Beach County 

P.O. Box 16568 

West Palm Beach, Fl 33416 



Dear Mr. Schindel, 

Below is the City's response to your audit recommendations in the November 8, 2012 
Draft Report on the Audit of the Municipal Public Safety Communications Consortium 
(MPSCC). These responses are to the recommendations that were specifically 
addressed to West Palm Beach. 

Recommendation H o. 3 : WPB should work with the MPSCC to plan and schedule 
another System Reliability test utilizing the standard Harris equipment configuration. 

Response: A proposal for P25 system was received by the City. 

Recommendation Ho. 12 : Before deciding whether to proceed with deploying Opensky 
in WPB, the MPSCC and WPB need to perform a full DAQ test with sufficient test points 
to identify all buildings that have a loss factor greater that 12db and determine what 
level of signal strength is needed to penetrate those denser buildings. 

Response: We concur 

Recommendation Ho. 13 : If the MPSCC and WPB decide to move forward with 
Opensky or another Harris system, a separate contract with Harris should be executed 
that includes specifications that meet WPB's more challenging infrastructure. The 
contract should also provide adequate protections for the MPSCC and WPB if system 
performance does not meet contract specifications. 

Response: We concur. 
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Recommendation No. 14: WPB should carefully consider each of their options. To 
assist in making the best decision, WPB should consider hiring an independent radio 
system consultant to thoroughly evaluate the City's needs, including their unique and 
more challenging infrastructure needs. 

Response: We concur. An independent consultant was hired in September 2102. 

Recommendation No. 15: WPB should have their radio Technical Team assess the 
timeframe left for replacing their current aging system so that can be factored into any 
decision on moving forward with any of their available options. 

Response: We concur 

Thank you for the opportunity to comment. 



Sincerely, 




Ed Mitchell, City Administrator 



'An Equal Opportunity Employer 
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Facilities Development & 
Operations Department 



2633 Vista Parkway 
West Palm Beach, FL 3341 ! 



Telephone -(561) 233-0200 
Facsimile -(561) 233-0206 
www.pbcgov.com/fdo 



Palm Beach County 
Board of County 
Commissioners 



Steven L, Abrams, Chairman 
Priscilla A. Taylor, Vice Chair 
Hal R. Valeche 
Paulette Burdick 

Shelley Vana 
Mary Lou Berger 
Jess R. Santamaria 

County Administrator 

Robert Weisman, P.E, 



"An Equal Opportunity 
Affirmative Action Employer" 



November 27, 2012 
To: 



From: 



Re; 



Dennis Schindel, Director of Audit 
Office of Inspector General 

Audrey Wolf, Director 

Facilities Development & Operations 

Draft Audit Report of OoenSkv Public Safety Radio System 



The County has the following management responses to two (2) findings of the 
draft audit report. 

Finding (13) 

The County was asked by City Staff to prepare a summary of the options for the 
City of West Palm Beach to connect to the County system, costs associated with 
those options, along a history of the County's historical and current (then) 
position on $12.50 distributions. The County advised the City Staff that any 
summary would merely be a broad description of the options and a recitation of 
the County costs offered to other cities basically on a unit basis. Because of the 
lack of specific City system requirements, the numbers contained in that 
presentation could not be used directly for any financial conclusions. The 
County costs provided could only be applied AFTER a thorough technical 
analysis ending in a definitive set of systems requirements. Further, the 
presentation would only be a summary and not reflect the variances which exist 
among the County options, nor the impact of timing, both of which impact the 
financial conclusions. 

The summary was prepared in the form of a power point presentation which 
was delivered to Staff for comment. The County never received any comments, 
nor was the presentation ever delivered by County or City Staff to the City 
Commission. 

The County agrees with Recommendations 14 and 15 which conclude Finding 
13, but disagrees with some of the direct statements made/financial and 
technical conclusions drawn and/or are implied by the findings leading up to 
those recommendations. The County would suggest that the recommendations 
be moved to another section and the finding deleted. Further, this finding does 
not seem to be within the stated scope/purpose of the audit. 
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Finding (18) 

Since the time that the CSOPAC sunset, the conclusions, policies and objectives have been re- 
affirmed multiple times including as recently as August 2012. As such, there is no basis or need 
for reconvening any type of CSOPAC-like committee. Further, there are several existing operational 
planning committees already in place which have, in whole or in part, overlapping memberships and/or 
communications system missions (ie: Chiefs of Police Association- Communications Sub-Committee, 
PSAP Committees, CSRCC - Law Enforcement User Group, CRSCC - Fire/Rescue EMS User Group, and 
CRSCC - Public Works User Group). The establishment of an additional planning committee will dilute 
the existing committees' attendance/success. 

However, information sharing is warranted so that each entity can plan its future. To that end, the 
County would agree to schedule, host and Staff an annual Public Safety Radio System 
Communications Information Sharing Workshop with the administrator, police chief and fire chief from 
each agency being invited to attend. The workshop would bring together all agencies and provide the 
various existing operational committees and provide a single opportunity for others that may routinely 
participate in the operational committees to receive/discuss status and issues at a single meeting. At a 
minimum, the agenda would include: 

1. Presentations by each agency operating a radio system regarding the current condition of its 
system, its funded initiatives, and planned future initiatives. 

2. Presentations by each planning committee regarding new operational objectives and required 
system functionality to meet those goals. 

3. Identification of follow-up action items for established committees. 

While certainly #1 is important, the amount of preparation and open participation of the existing 
operational committees is key to making the workshop successful. 

If you have any questions or would like to discuss further, please do not hesitate to call. 



c: Robert Weisman, County Administrator 
Nancy Albert, Director ESS 

Mark Filla, ESS/Public Safety Radio System Administrator 
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Municipal Public Safety Communications Consortium 
Of Palm Beach County, Inc* 
A Florida Not-for-Profit Corporation 
360 Orange Tree Drive 
Atlantis, Florida 33462 

Phone: (561) 965-1700 ♦ Fax: {561) 968-9443 



MPSGC 



Chairman 
Robert G. Mangold 



November 28, 2012 



Dennis Schindel 

Director of Audit 

Office of the Inspector General 

PO Box 16568 

West Palm Beach, FL 33416 



Dear Mr. Schindel, 

The Municipal Public Safety Communications Consortium (MPSCC) is pleased to review and 
provide a response to the inspector General's Office (iG) evaluation of the sourcing and 
implementation of the interoperable countywide public safety radio system. The members who 
are implemented of the MPSCC - Atlantis, Juno Beach, Jupiter, Palm Beach, Palm Beach 
Gardens, Jupiter Inlet Colony, and the School Board - are in general agreement with the report, 
and are pleased to read that the ultimate findings report a functional, interoperable system with 
good performance and validate that there were no improper expenditures. 

The implemented consortium members agree with the IG's assessment that the MPSCC has 
accomplished the task of successfully installing a system that performs to contract 
specifications. The MPSCC is also pleased that the IG's report notes that the system test 
results to-date appear better than many other similar efforts around the country. Further, the 
MPSCC appreciates the acknowledgement that we worked well with each other, and with 
Harris, to address issues during implementation that ultimately yielded a positive result 

Several areas for improvement were proposed by the IG in the report related to the 
management of the consortium. The MPSCC welcomes the IG's suggestions for continuous 
improvement and is working toward solutions to many of the recommendations. In fact, in a 
separate evaluation, the MPSCC's own ad-hoc governance committee identified some of the 
same issues last summer. 

Taking all points of feedback in the report as constructive, the members of the MPSCC 
appreciate the thorough evaluation summarized in the IG report. The members who have 
implemented the system stand ready to work together and with other agencies and 
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municipalities toward an interoperable public safety radio system. It is the goal of the 
consortium to ensure that systems work across agencies seamlessly and safely, and that any 
technological upgrades or enhancements are implemented in a fiscally-responsible way. 



Sincerely, 



Members of the Municipal Public Safety Communications Consortium (MPSCC) implementing 
the Open Sky public safety radio system: 



Mo Thornton, City Manager, City of 
Atlantis 



Andrew D. Lukasik, Town Manager, Town 
of Jupiter 

Joseph F. Lo Bello, Town Manager, Town 
of Juno Beach 

Peter Elwell, Town Manager, Town of 
Palm Beach 

Ron Ferris, City Manager, City of Palm 
Beach Gardens 



Chief Robert G, Mangold, Chairman, 
MPSCC, City of Atlantis Police 
Department 

Chief Frank J. Kitzerow, Town of Jupiter 
Police Department 

Chief Brian Smith, Town of Juno Beach 
Police Department 

Kirk W. Blouin, Director of Public Safety, 
Palm Beach Police Department 

Chief Stephen J. Stepp, City of Palm 
Beach Gardens Police Department; 

Colonel Ernie Carr, Executive Director, 
MPSCC, Palm Beach Gardens Police 
Department 



Sincerely, 

Robert G. Mangold, Chai 
Municipal Public Safety Communications Consortium 




Page 2 of 2 



Page 55 



Office of Inspector General 



Audit # 201 3-A-0001 



ATTACHMENT 3 (Continued) 



Office of Inspector General Recommendations 



#1 The MPSCC should ensure that sufficient funding exists before proceeding with any procurement and 
especially before awarding a contract. 

Response: The MPSCC will continue to provide the proper procurement practices by requiring funding 
to be established prior to purchase or contract for purchase. 

#2 The MPSCC and West Palm Beach management should work together to determine whether 
OpenSky can provide the most cost effective solution for the WPB's non-public safety radio 
communication needs. 

Response: The MPSCC acknowledges this recommendation, that every aspect of West Palm Beach's 
radio system needs both public safety and nonpublic safety be established and provide a comprehensive 
solution including cost. 

#3 WPB should work with the MPSCC to plan and schedule another system reliability test utilizing 
standard Harris equipment configuration. 

Response: A proposal for P25 system was received by the City of West Palm Beach. 

#4 The MPSCC needs to ensure for future acquisitions or any expansion of the current OpenSky system, 
that all acceptance testing required by contract is completed and fully documented. 

Response: The MPSCC will arrange for that testing in the future, either new or retesting, that a more 
extensive and comprehensive documentation approach be required by contract. 

#5 For all future contracts the MPSCC should ensure contract terms and conditions are sufficient to 
adequately protect public funds. 

Response: In the future, milestones in MPSCC contracts will provide for more extensive validation of 
functionality, both in the development of the milestone and in its essential value based on the dollar 
amount of the milestone payment. 

#6 When contract deliverables are changed during the execution of a contract a contract amendment or 
change order must be issued and determination made whether the contract cost should be adjusted. 

Response: The MPSCC agrees that change order forms be executed in the future and continue to take 
contract changes before the Board of Directors for discussion and approval. 

#7 The MPSCC must ensure that they receive adequate supporting documentation of all contract 
deliverables before contract payments are made. 
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Response: The MPSCC agrees with this recommendation. 

#8 All MPSCC purchases made for additional OpenSky hardware and/or software should follow the 
terms of the contract with M/A-Com (Harris) and be purchased by contract change order. 

Response: The MPSCC agrees with this recommendation. 

#9 The MPSCC should establish a process to document and evaluate all failed indoor testing locations 
even those that exceed contract specifications to determine if they pose a significant officer and public 
safety risk that needs to be resolved. Officers should be periodically reminded to document report 
locations where radio communication failures occurred so that they can be recorded, mapped and 
evaluated to determine if additional fixes are needed. 

Response: Separate and distinct documentation identifying structures that exceed contract 
specifications for building radio signal penetration in the future will be identified as part of the 
acceptance test planning (ATP) and made part of the testing protocol, to evaluate the density of the 
building and determine penetration values of that building for acceptable radio communications. Note, 
as a standard, buildings that are known to exceed the penetration value guaranteed by contract are not 
considered a failure of the contract, nor will be in the future, it will be documented for officer safety. 
Other solutions implemented for building coverage will be determined as necessary by the particular 
member agencies. 

#10 The MPSCC needs to focus on identifying the cause(s) and resolving the three major problem areas 
identified by users in this survey as dead spots, audio quality, and dropped calls. Additional system 
testing may be needed in these three municipalities to determine if additional infrastructure is needed 
to improve coverage. 

Response: This is a process that we regularly perform to maintain the integrity of the system. We 
regularly survey officers and solicit their input for any areas that they may identify with communication 
problems, have fixed those problems, and will continue to do so. 

#11 The MPSCC needs to implement a formal incident resolution system (helpdesk) to address any 
concerns raised by the users of OpenSky. The system should follow a standard set of technical system 
guidelines (such as ITIL 8 ) to document and resolve all incidents and/or problems. This system should 
include mapping of any reported coverage problems to determine if certain locations or structures are 
chronic problem areas that need to be corrected. 

Response: Through the teamwork with individual municipalities developing in-house 
policies/procedures, to achieve this recommendation will be put into place within the next 120 days. 

#12 Before deciding whether to proceed with deploying OpenSky in West Palm Beach, the MPSCC and 
WPB need to perform a full DAQtest with sufficient test points to identify all buildings that have a loss 
factor greater than 12 dB and determine what level of signal strength is needed to penetrate those 
denser buildings. 
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Response: West Palm Beach should perform a full DAQtest as suggested by the Inspector General. 

#13 If the MPSCC and West Palm Beach decide to move forward with OpenSky or another Harris 
system, a separate contract with Harris should be executed that includes specifications that meet WPB's 
more challenging infrastructure. The contract should also provide adequate protections for the MPSCC 
and West Palm Beach if system performance does not meet contract specifications. 

Response: Should the City of West Palm Beach move forward with OpenSky or another Harris system an 
amendment to the Interlocal agreement must be completed with the MPSCC and a contract with Harris. 

#14 West Palm Beach should carefully consider each of their options. To assist in making the best 
decision, WPB should consider hiring an independent radio systems consultant to thoroughly evaluate 
the city's needs, including their unique and more challenging infrastructure. 

Response: West Palm Beach has hired a consultant to evaluate their options. 

#15 West Palm Beach should have their radio Technical Team assess the timeframe left for the replacing 
their current aging system so that can be factored into any decision on moving forward with any of their 
available options. 

Response: MPSCC concurs. 

#16 The MPSCC should begin planning system replacement and consider collecting additional funds for 
members to establish a separate reserve account to adequately fund future system replacement cost. 

Response: Within the next year the M PSCC will develop a program to sufficiently address the 
recommendation suggested by the Office of Inspector General 

#17 We recommend that the MPSCC arrange for an annual financial audit. The annual audit is an 
essential component to show the MPSCC's accountability for contributions and expenditures according 
to the mission of the organization. The annual audit should be conducted by an independent audit firm 
and audit fees should be budgeted accordingly. 

Response: The current budget (2012/2013) established by the MPSCC contains funding for such an audit 
to be conducted. 

#18 The Board of Directors should also consider hiring professional accounting personnel or competent 
third-party to manage MPSCC's accounting activities and prepare the annual financial statements. 
Structured and timely accounting should be established for the long run to handle the substantial funds 
from/to various sources. 

Response: Currently a scope of work is being developed, after which, through a proper procurement 
bid, personnel will be hired based on the job description and scope of work developed. Projected 
timeline is 120 to 180 days. 



Page 58 



Office of Inspector General 



Audit # 201 3-A-0001 



ATTACHMENT 3 (Continued) 



#19 There should be a Board agreement/policy concerning budget and budget to actual reviews, and the 
purpose and the use of operating reserves. This policy should define and set goals for reserve funds, 
described authorization for use of reserves and outline requirements for reporting and monitoring. 

Response: The board will develop a policy documenting procedures for the above recommendation 
within the next 120 days. 

#20 The MPSCC needs to establish a policy for fixed assets including the use of fixed asset tags. 

Response: The board will develop a policy documenting procedures for the above recommendation 
within the next 120 days. The MPSCC, at the suggestion of Inspector General, already started the fixed 
asset tagging process and will be completed within the next 90 days. 

#21 The MPSCC needs to establish travel policies including types and amounts of allowable expenses. 
Response: Policy will be established within the next 90 days. 
#22 MPSCC should establish a document retention policy. 
Response: Policy will be established within the next 90 days. 

#23 The organization should have policies establishing for segregation of duties of key transactions. The 
person who records the transactions should not be the person also verifies the delivery of goods. The 
duties of authorization, recording, custody and reconciliation should be segregated. 

Response: The formal policy will be developed within the next 120 days; however implementation of 
that policy is currently being utilized to achieve the best segregation of duties possible. 

#24 The MPSCC should consider hiring administrative staff to ensure that all of additional policies 
procedures and processes that are needed to be put into place get timely established and are 
consistently maintained and carried out going forward. 

Response: As mentioned in recommendation 18, by the OIG (additional personnel), we are currently 
preparing the scope of work (under development). 

#25 The governing board of the MPSCC should ensure that all activities are in compliance with internal 
control standards. 

Response: The MPSCC agrees; and quarterly meetings are already in the planning and will be 
established for compliance review, within the next 90 days. 

#26 Establish a countywide public safety radios communications committee to ensure coordination and 
compatibility between all organizations involved in implementation, support, and/or use of public safety 
radios. 

Response: The MPSCC believes that the once established and disbanded Countywide Public Safety Radio 
Communications Committee and Communications Systems and Operations Policy Advisory Committee 
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(CSOPAC) before disbandment in 2001 allowed for dialogue and views for public safety communications 
to at least be shared for discussion and for familiarization of other systems. Hopefully a new committee 
can be formed so that proposed radio planning and philosophies for the future can be shared. We 
wholeheartedly support the development of this committee. 



Findings 

The Office of Inspector General has produced a 47 page report stating their findings and 
recommendations. The MPSCC has responded to all recommendations from the IG. The findings are 
generally adequately documented with some degree of certification but there are areas in the findings 
that are subjective and arbitrary as to the IG's opinions, and lacks sufficient detail and analysis showing 
cause-and-effect, in some instances. 

As an example, the IG puts their emphasis on the lack of management, when in fact, there were many 
uncontrollable factors that impacted the delay of installation as briefly reviewed below. 

IG document page 2 paragraph 1 management 

The original target date of March 21, 2007 for completion and acceptance of phase 1 
was significantly exceeded. The first of the three phases 1 municipalities, Palm Beach 
Gardens (PBG), did not become operational until September 2009 in the second, Palm 
Beach (PB) until April 2010, The third, WPB is still not operational. System acceptance 
occurred on July 20, 2010 over 40 months after the original target date. 

MPSCC Response: 

Management, or the lack of, was not the primary cause for delay. Tower site acquisition 
was the salient cause; the last tower site was brought online in 2009. 

The timing of Town of Palm Beach coming live on the system was not a project 
management issue, but designed. The Town waited for the legacy system 
recpjrfjg^BtjpiLj^ which would provide backup for officer safety and 

system redundancy. 

Management played only a small part of delaying the system acceptance in July 201 0. 
As stated, tower site acquisition, the inability to establish a tower site in IBIS took many 
months. The use of West Palm Beach's wastewater tower site was challenged and 
delayed by Palm Beach County for many months. This represents only some of the 
issues encountered by the MPSCC during implementation. However, the underlying 
cause for the system delay was in fact tower site acquisition. 
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